Validates that package metadata signals -- especially difficult-to-manipulate temporal and community-interaction features like package age, contributor count, and stakeholder service time -- reliably distinguish malicious from benign npm packages.
Halder et al. (WWW '24) present MeMPtec, a metadata-based malicious package detection model for the npm ecosystem that partitions extracted metadata features into easy-to-manipulate (ETM) and difficult-to-manipulate (DTM) categories based on monotonicity and restricted-control properties. ETM features include binary indicators like whether an author email, license, homepage, GitHub link, or description exists, plus name length and special character presence. DTM features include package age, modification duration, stakeholder service time, community contribution score (log-scaled product of service time and contributed-package count), stars, forks, subscriber count, pull requests, and issue count. Trained on 3,232 malicious and up to 32,320 benign npm packages across five ML/DL algorithms (SVM, GBM, GLM, DRF, ANN), MeMPtec achieved 99.97% accuracy (DRF, imbalanced) and reduced false positives by an average of 97.5% and false negatives by an average of 80.42% on imbalanced data compared to the existing feature baseline. Under adversarial simulation where feature values of malicious packages were replaced with benign distributions in decreasing importance order, MeMPtec retained 92.73% accuracy (GLM) even after 100% feature manipulation, whereas the existing baseline dropped to approximately 50%. DTM temporal features declined only marginally after 360 days of simulated aging (0.9998 to 0.9928 accuracy). The paper notes that packages with unknown authors are likely malicious, and that an attacker connection between maintainer count and potential for injecting malicious code has been demonstrated in prior work (Zimmermann et al.).
MeMPtec was trained on 3,232 confirmed malicious npm packages from the Backstabber's Knife Collection dataset and achieved 99.97% accuracy detecting active malware using metadata signals alone.
installing a package that contains active malicious code such as credential theft, data exfiltration, or cryptocurrency mining
the study demonstrates that metadata patterns of confirmed malicious packages (missing author info, zero community engagement, very young package age) are statistically distinct from benign packages, confirming that metadata-based detection reliably surfaces active malware
Package age is classified as a monotonic DTM feature -- it can only increase over time and cannot be faked. MeMPtec's adversarial analysis shows that even after 360 days of simulated temporal manipulation, model accuracy declined only from 0.9998 to 0.9928, confirming package age as a robust predictor of maliciousness.
adopting a newly created package that lacks the time-accumulated community vetting needed to surface hidden malicious behavior
the study's adversarial robustness experiments demonstrate that package age is one of the most manipulation-resistant features for distinguishing malicious from benign packages, as attackers cannot retroactively age a repository
Contributor count and stakeholder community contribution score are classified as DTM features. The paper cites Zimmermann et al.'s finding that the number of package maintainers is directly connected to the potential for introducing malicious code, and MeMPtec uses contributor_CPN and contributor_CCS as key predictive features.
depending on a package where a single malicious actor can inject harmful code without independent peer review
the study demonstrates that legitimate packages accumulate genuine contributors over time while malicious packages almost never do, making low contributor count a reliable proxy for elevated supply chain risk
MeMPtec identifies stakeholder service time and contributed package number as DTM features, and the paper states that 'a package that has unknown authors is likely to be malicious' -- single-contributor packages with short service times concentrate the malicious-package risk signals.
relying on a package maintained by a single individual who may be an attacker or whose account could be socially engineered for a supply chain compromise
the study finds that malicious packages are statistically characterized by having a single stakeholder with minimal community history, making sole-contributor status a strong metadata signal for maliciousness
MeMPtec classifies the existence of GitHub link, homepage link, bugs link, and issues link as ETM features, and uses the associated community signals (stars, forks, issues, pull requests) as DTM features. A package without a verifiable source repository scores zero on all these DTM features, placing it squarely in the malicious cluster.
consuming a package whose source code cannot be independently verified, preventing detection of injected malicious code
the model shows that packages without accessible source repositories lack the community-interaction DTM features (stars, forks, contributors) that characterize legitimate packages, making missing repositories a strong indicator of elevated risk
MeMPtec includes scripts_exist and scripts_length as ETM features extracted from package metadata. The paper notes that malicious packages use scripts as an attack vector, referencing the event-stream incident where a malicious package was introduced into a popular npm dependency.
executing arbitrary code on a developer's machine during package installation before the consumer has any opportunity to review the package contents
the study's feature extraction treats install script presence and length as predictive indicators of maliciousness, consistent with the known attack pattern where malicious npm packages use install hooks to run exploit code immediately upon installation
MeMPtec uses community-interaction signals (stars, forks, subscriber count, pull request count, issue count) as restricted-control DTM features that an adversary cannot directly manipulate, and these features are among the top predictors of maliciousness.
Risk Guard does not incorporate GitHub social signals (stars, forks, subscriber count) or activity-based engagement metrics (pull request volume, issue volume) as package risk indicators.
Risk Guard would be better if it incorporated community engagement metrics such as star count, fork count, and pull request volume as difficult-to-fake signals of project legitimacy.
MeMPtec classifies the existence and length of package descriptions, readmes, keywords, and homepage links as predictive ETM features -- packages missing these metadata fields are more likely to be malicious.
Risk Guard does not check for the completeness of package metadata fields such as description, readme, or keywords in the registry.
Risk Guard would be better if it flagged packages with sparse or missing registry metadata (no description, no readme, no keywords) as an indicator of reduced trustworthiness.