A proposed change to the Python packaging system could significantly improve the way dependencies are tracked and vulnerabilities are analyzed. A Python Enhancement Proposal (PEP) recently floated on python.org suggests incorporating Software Bill-of-Materials (SBOM) documents into Python packages to address the issue of "phantom dependencies" and enhance the "measurability" of Python packages.

The proposal, created on January 2, acknowledges that Python packages are particularly affected by the phantom dependency problem. This occurs when packages include software components not written in Python, such as compiled libraries from C, C++, Rust, Fortran, and other languages, for reasons like compatibility with standards, ease of installation, or use cases like machine learning. The Python wheel format, preferred by users for its ease of installation, requires bundling shared compiled libraries without a method to encode metadata about them. As a result, packages related to Python packaging sometimes need to solve the bootstrapping problem, including pure Python projects inside source code, which cannot be described using Python package metadata.

The inclusion of an SBOM document annotating all included libraries would enable Software Composition Analysis (SCA) tools to reliably identify the included software. SBOM is a technology-and-ecosystem-agnostic method for describing software composition, provenance, heritage, and more, making it an ideal solution for improving the measurability of Python packages. Moreover, SBOMs are used as inputs for SCA tools, such as scanners for vulnerabilities and licenses, and are required by recent security regulations, like the Secure Software Development Framework (SSDF).

The proposal suggests delegating SBOM-specific metadata to SBOM documents included in Python packages and adding a core metadata field for discoverability of included SBOM documents. This change would enable SCA tools to accurately identify and report on vulnerable software components, addressing a significant gap in the current Python packaging system.

The demand for SBOM documents of open-source projects is expected to remain high due to recent security regulations. By incorporating SBOM documents into Python packages, the Python community can improve the reliability and security of its packages, making it easier for developers to track dependencies and identify vulnerabilities.

The proposed change has significant implications for the Python ecosystem, particularly in the context of security and compliance. As the Python community continues to grow and evolve, it is essential to address the phantom dependency problem and improve the measurability of Python packages. The incorporation of SBOM documents into Python packages could be a crucial step towards achieving this goal.

As the proposal moves forward, it will be interesting to see how the Python community responds to this change and how it will impact the development of Python packages. One thing is certain – the inclusion of SBOM documents in Python packages has the potential to significantly enhance the security and reliability of the Python ecosystem.