DPM - Dependency Package Manager

A cross-language package manager written in Python that resolves dependencies using a hybrid greedy/backtracking algorithm.

What it does

DPM is a cross-language dependency package manager that solves the complex problem of dependency resolution across multiple package ecosystems. It intelligently:

Fetches packages from PyPI (Python), npm (JavaScript), and system package managers (apt, yum, brew)
Resolves dependencies by finding compatible versions that satisfy all constraints
Handles conflicts using a hybrid greedy/backtracking algorithm
Installs packages in the correct order with integrity verification
Manages projects with manifest files and lock files for reproducibility

The Problem DPM Solves

When you want to install multiple packages, you often run into "dependency hell":

Package A requires numpy >= 1.20.0
Package B requires numpy < 1.22.0
Package C requires numpy == 1.21.0

DPM automatically finds a version that satisfies all constraints (in this case, 1.21.0). If no solution exists, it provides detailed conflict information to help you resolve the issue.

Why Use DPM?

Cross-language: Manage Python, JavaScript, and system packages in one tool
Smart resolution: Handles complex dependency conflicts automatically
Fast: Uses greedy algorithm for 90% of cases (< 1 second)
Complete: Falls back to backtracking for complex scenarios
Reproducible: Lock files ensure consistent installs across environments
Robust: Built-in retry logic, validation, and error recovery
Production-ready: Comprehensive error handling and logging

Quick Start

# install it
git clone https://github.com/yomnahisham/dpm.git
cd dpm
pip install -e .

# or use directly
python3 -m dpm.main install requests flask

# try it out
dpm install requests flask
dpm tree requests
dpm info numpy
dpm search flask

Installation

Requirements

Python 3.8+
pip (for installing DPM itself)

Install DPM

# clone the repository
git clone https://github.com/yomnahisham/dpm.git
cd dpm

# install in development mode
pip install -e .

# or install globally
pip install .

Verify Installation

dpm --version
dpm --help

Commands

Command	Description	Example
`install`	install packages	`dpm install numpy pandas`
`remove`	uninstall packages	`dpm remove flask`
`update`	update to latest versions	`dpm update requests`
`list`	show installed packages	`dpm list`
`resolve`	dry run - show what would install	`dpm resolve django`
`tree`	show dependency tree	`dpm tree flask`
`info`	show package details	`dpm info requests`
`search`	find packages	`dpm search flask`
`lock`	generate lock file	`dpm lock requests flask`
`venv`	manage virtual environment	`dpm venv create`
`init`	initialize dpm.json manifest	`dpm init myproject 1.0.0`
`clean`	remove unused packages	`dpm clean`
`outdated`	check for outdated packages	`dpm outdated`
`cache`	manage cache	`dpm cache info`
`pin`	pin package version	`dpm pin requests@2.32.5`
`unpin`	unpin package	`dpm unpin requests`
`export`	export dependencies	`dpm export requirements.txt`
`repo`	manage repositories	`dpm repo list`

Command Options

--verbose, -v: Show detailed output
--debug, -d: Show debug information (includes verbose)
--offline: Use cache only, no network requests
--skip-integrity: Skip integrity verification
--show-resolution: Show detailed resolution steps

Manifest File (dpm.json)

DPM uses a dpm.json file to track project dependencies:

{
  "name": "my-project",
  "version": "1.0.0",
  "dependencies": {
    "requests": "^2.32.0",
    "numpy": ">=1.20.0"
  },
  "devDependencies": {},
  "sources": ["pypi", "npm"]
}

Initialize a new project:

dpm init myproject 1.0.0

Lock Files

DPM creates a dpm.lock file to ensure reproducible installs:

dpm lock requests flask    # creates dpm.lock
dpm install                # installs from lock file

The lock file contains exact versions and SHA256 checksums for integrity verification.

How the Algorithm Works

DPM uses a hybrid approach that combines speed and completeness. It tries the fast greedy algorithm first, and only falls back to backtracking when conflicts are detected.

The Resolution Process

Parse dependencies: Extract version constraints from all packages
Build dependency graph: Create a graph of package relationships
Try greedy resolution: Fast path for simple cases
Fall back to backtracking: If conflicts detected, use complete solver
Generate installation plan: Order packages by dependencies
Install with verification: Install packages and verify integrity

1. Greedy Resolver (Fast Path)

The greedy algorithm processes packages in topological order (dependencies before dependents):

for each package in topological order:
    select the best version that satisfies all constraints
    - prefer stable versions over prereleases
    - prefer latest version
    - prefer versions with fewer dependencies
    if conflict detected:
        return failure (trigger backtracking)
    propagate constraints to dependents

Characteristics:

Time complexity: O(V + E) where V = packages, E = dependencies
Success rate: ~90% of real-world cases
Speed: Typically < 1 second for most packages
Limitation: Cannot handle all conflict scenarios

Example:

Request: install flask, django

Greedy process:
1. flask → select 3.1.2 (latest stable)
2. django → select 6.0 (latest stable)
3. Check dependencies:
   - flask needs: blinker, click, jinja2, werkzeug
   - django needs: asgiref, sqlparse, tzdata
4. No conflicts → success! (took < 1 second)

2. Backtracking Resolver (Complete Solver)

When greedy fails, backtracking systematically explores the version space:

def backtrack(remaining_packages, assignments):
    if no remaining packages:
        return success
    
    # MRV heuristic: pick package with fewest valid versions
    package = select_most_constrained(remaining_packages)
    
    for each valid version of package:
        if forward_check(package, version, remaining_packages):
            assignments[package] = version
            if backtrack(remaining_packages - package, assignments):
                return success
            # backtrack: unassign and try next version
            del assignments[package]
    
    return failure

Optimizations:

MRV (Minimum Remaining Values): Tries packages with fewer options first
Forward checking: Prunes invalid versions early
Constraint propagation: Detects dead-ends before exploring them
Memoization: Caches failed states to avoid redundant work

Characteristics:

Time complexity: Worst case O(b^d) where b = branching factor, d = depth
In practice: Usually much faster due to pruning and memoization
Completeness: Guaranteed to find a solution if one exists
Speed: Typically 1-5 seconds for complex cases

Example:

Request: install package-a, package-b

Conflict detected:
- package-a@1.0.0 needs dependency-x@>=2.0.0
- package-b@2.0.0 needs dependency-x@<2.0.0

Backtracking process:
1. Try package-a@1.0.0, package-b@1.0.0
   - Check if dependency-x exists that satisfies both
   - No valid version → backtrack
2. Try package-a@0.9.0, package-b@2.0.0
   - Check constraints
   - Success! → return solution (took ~2 seconds)

Why Hybrid?

The hybrid approach gives you the best of both worlds:

Approach	Speed	Completeness	Use Case
Greedy only	Very fast	Incomplete	Simple projects
Backtracking only	Slow	Complete	Complex projects
Hybrid	Fast (90% of time)	Complete	All projects

Real-world performance:

Simple cases (1-3 packages): < 1 second (greedy)
Medium cases (4-10 packages): 1-3 seconds (greedy)
Complex cases (conflicts): 1-5 seconds (backtracking)
Large trees (100+ packages): 5-15 seconds (usually greedy)

Conflict Detection and Reporting

When resolution fails, DPM provides detailed information:

Conflict detected:
- package-a@1.0.0 requires dependency-x@>=2.0.0
- package-b@2.0.0 requires dependency-x@<2.0.0

Constraint chain:
  package-a@1.0.0
    → dependency-x@>=2.0.0
  package-b@2.0.0
    → dependency-x@<2.0.0

No valid version of dependency-x satisfies both constraints.

This helps you understand why resolution failed and how to fix it.

Package Sources

DPM supports multiple package sources, allowing you to manage dependencies across different ecosystems:

Source	Language	API	Example
PyPI	Python	`pypi.org/pypi/{pkg}/json`	`dpm install requests`
npm	JavaScript	`registry.npmjs.org/{pkg}`	`dpm install express`
System	varies	apt/yum/brew commands	`dpm install git`
Local	any	JSON files	For testing/development

How Sources Work

Each source implements a common interface:

fetch_latest(package_name) - Get latest version
fetch_version(package_name, version) - Get specific version
get_dependencies(package_name, version) - Get dependency list
package_exists(package_name) - Check if package exists
search(query, limit) - Search for packages
prefetch(names) - Parallel fetching for performance

Source Priority

When multiple sources are configured, DPM checks them in order:

PyPI (for Python packages)
npm (for JavaScript packages)
System (for system packages)
Local (for testing)

You can configure which sources to use in dpm.json:

{
  "sources": ["pypi", "npm", "system"]
}

Project Structure

dpm/
├── core/              # basic data types
│   ├── package        # package metadata
│   ├── version        # semver parsing and comparison
│   ├── dependency     # version constraints
│   ├── manifest       # dpm.json handling
│   ├── config         # configuration management
│   ├── logger         # structured logging
│   ├── exporter       # export to other formats
│   └── repository     # custom repository management
├── resolver/          # the algorithm stuff
│   ├── resolver       # main entry point, hybrid logic
│   ├── greedy         # fast greedy solver
│   ├── backtrack      # csp-style backtracking
│   └── graph          # dependency graph, cycle detection
├── sources/           # where we get packages from
│   ├── source         # base source interface
│   ├── pypi           # python packages
│   ├── npm            # javascript packages
│   ├── system         # apt/yum/brew
│   └── local          # json files for testing
├── installer/         # actually installs stuff
│   ├── installer      # calls pip/npm/apt
│   ├── plan           # installation ordering
│   ├── state          # tracks what's installed
│   ├── lockfile       # dpm.lock handling
│   └── venv           # virtual environment management
├── network/           # http and caching
│   ├── http_client    # urllib wrapper with parallel fetching
│   └── cache          # disk cache for api responses
└── cli/               # command line interface
    └── commands       # install, remove, list, etc

Usage Examples

Basic Workflow

# 1. Initialize a project
dpm init myproject 1.0.0

# 2. Install packages
dpm install requests flask

# 3. View dependency tree
dpm tree flask

# 4. Create lock file for reproducibility
dpm lock requests flask

# 5. Install from lock file (exact versions)
dpm install

Dependency Tree Visualization

$ dpm tree flask

Dependency tree:

`-- flask@3.1.2
    |-- blinker@1.9.0
    |-- click@8.3.1
    |   `-- colorama@0.4.6
    |-- importlib-metadata@8.7.0
    |   |-- zipp@3.23.0
    |   `-- typing-extensions@4.15.0
    |-- itsdangerous@2.2.0
    |-- jinja2@3.1.2
    |   `-- markupsafe@3.0.3
    `-- werkzeug@3.1.2
        `-- markupsafe@3.0.3

Resolution Output

$ dpm resolve requests flask

Resolving dependencies for: requests, flask

Resolved 17 packages:
  * MarkupSafe@3.0.3
  * blinker@1.9.0
  * certifi@2025.11.12
  * charset_normalizer@3.4.4
  * click@8.3.1
  * colorama@0.4.6
  * flask@3.0.3
  * idna@3.11
  * importlib-metadata@8.7.0
  * itsdangerous@2.2.0
  * jinja2@3.1.4
  * markupsafe@3.0.3
  * requests@2.2.0
  * typing-extensions@4.15.0
  * urllib3@1.26.20
  * werkzeug@3.1.4
  * zipp@3.23.0

Package Information

$ dpm info requests

Package: requests
Version: 2.32.5
Language: python
Source: PyPI

Dependencies (4):
  * charset_normalizer<4.0.0,>=2.0.0
  * idna<4.0.0,>=2.5.0
  * urllib3<3.0.0,>=1.21.1
  * certifi>=2017.4.17

Advanced Usage

# Search for packages
dpm search json

# Pin a package to exact version
dpm pin requests@2.32.5

# Check for outdated packages
dpm outdated

# Export to requirements.txt
dpm export requirements.txt

# Use offline mode (cache only)
dpm --offline resolve requests

# Verbose output for debugging
dpm --verbose resolve flask django

Configuration

DPM uses configuration files for customization:

User config: ~/.dpm/config.json
Project config: dpm.config.json (optional)

Example config:

{
  "cache_dir": "~/.dpm/cache",
  "default_sources": ["pypi", "npm"],
  "timeout": 30,
  "max_workers": 4,
  "log_level": "INFO",
  "cache_ttl_hours": 24,
  "cache_max_size_mb": 100,
  "resolution_timeout_seconds": 60,
  "retry_attempts": 3,
  "retry_backoff_factor": 0.5
}

Robustness Configuration

DPM includes several robustness features that can be configured in ~/.dpm/config.json:

Setting	Default	Description
`cache_ttl_hours`	24	How long cached data is valid before expiration
`cache_max_size_mb`	100	Maximum cache size before automatic eviction
`resolution_timeout_seconds`	60	Maximum time for dependency resolution
`retry_attempts`	3	Number of retries for network requests
`retry_backoff_factor`	0.5	Exponential backoff multiplier (0.5s, 1s, 2s)
`request_timeout`	30	Timeout per HTTP request in seconds

Why these defaults?

24h cache TTL: Balances freshness with performance (most packages don't change daily)
100MB cache: Large enough for thousands of packages, small enough to not fill disk
60s resolution timeout: Prevents hangs while allowing complex resolutions
3 retries: Handles transient network issues without excessive delays
0.5 backoff: Exponential backoff prevents overwhelming servers

Virtual Environments

DPM supports multiple virtual environment types:

# create a venv
dpm venv create myenv

# detect existing environments
dpm venv detect

# use existing environment
dpm venv use conda
dpm venv use poetry

# check status
dpm venv status

Export/Import

Export dependencies to other formats:

# export to requirements.txt
dpm export requirements.txt

# export to package.json
dpm export package.json

# export lock file
dpm export lock

Repository Management

DPM supports custom package repositories for private registries, mirrors, and alternative package sources.

Security Note: Credentials are stored in ~/.dpm/repositories.json with restricted permissions (600). However, passwords are stored in plain text. For better security:

Use API tokens instead of passwords when possible
Ensure ~/.dpm/repositories.json has proper permissions (chmod 600)
Avoid committing this file to version control
Consider using environment variables for sensitive credentials

Use Cases

Private Package Registries: Connect to your company's internal package registry
```
dpm repo add company-pypi https://pypi.company.com
```

Authenticated Repositories: Access private packages with credentials

# Recommended: Use API tokens instead of passwords
dpm repo add private https://private.com/repo token ""

# Alternative: Use username/password (stored in plain text)
dpm repo add private https://private.com/repo username password

Local Mirrors: Use faster or local package mirrors
```
dpm repo add local-mirror http://localhost:8080
```

Commands

# list all configured repositories
dpm repo list

# add a repository
dpm repo add myrepo https://example.com/repo

# add authenticated repository
dpm repo add private https://private.com/repo username password

# remove repository
dpm repo remove myrepo

Repositories are stored in ~/.dpm/repositories.json and will be used for package resolution in future releases.

Performance

DPM is optimized for speed and efficiency:

Resolution Performance

Scenario	Time	Algorithm
Single package	< 1s	Greedy
Multiple packages (2-3)	< 3s	Greedy
Complex resolution	1-5s	Backtracking
Large trees (100+ packages)	5-15s	Usually Greedy

Caching Performance

First run: Fetches from network (slower)
Subsequent runs: Uses cache (much faster)
Cache hit rate: ~95% for repeated operations

Optimization Techniques

Parallel Fetching: Fetches multiple packages simultaneously
Smart Caching: TTL and size limits prevent stale data
Early Termination: Stops as soon as solution is found
Memoization: Caches failed states to avoid redundant work
SystemSource Optimization: Caches system package checks

Use Cases

1. Python Projects

# Initialize Python project
dpm init myapp 1.0.0

# Install dependencies
dpm install flask sqlalchemy pytest

# Create lock file
dpm lock

# Install from lock file (reproducible)
dpm install

2. JavaScript Projects

# Install npm packages
dpm install express lodash

# Export to package.json
dpm export package.json

3. Mixed Projects

# Install from multiple sources
dpm install requests express git

# View dependency tree
dpm tree requests

4. CI/CD Pipelines

# Install from lock file (reproducible builds)
dpm install

# Verify integrity
dpm outdated  # should show no updates if lock file is current

Getting Help

Check CLI Reference for command details
See Architecture for how it works
Review Testing Guide for examples
Read Robustness Guide for error handling

Contributing

Contributions are welcome! Areas for improvement:

Additional package sources (RubyGems, Maven, etc.)
Performance optimizations
Additional test coverage
Documentation improvements

License

MIT License - see LICENSE file for details.

Name		Name	Last commit message	Last commit date
Latest commit History 12 Commits
benchmarks		benchmarks
cpp_implementation		cpp_implementation
docs		docs
dpm.egg-info		dpm.egg-info
dpm		dpm
tests/unit		tests/unit
.gitignore		.gitignore
LICENSE		LICENSE
PROJECT_REPORT.md		PROJECT_REPORT.md
README.md		README.md
dpm.json		dpm.json
requirements.txt		requirements.txt
setup.py		setup.py

Folders and files

Latest commit

History

Repository files navigation

DPM - Dependency Package Manager

What it does

The Problem DPM Solves

Why Use DPM?

Quick Start

Installation

Requirements

Install DPM

Verify Installation

Commands

Command Options

Manifest File (dpm.json)

Lock Files

How the Algorithm Works

The Resolution Process

1. Greedy Resolver (Fast Path)

2. Backtracking Resolver (Complete Solver)

Why Hybrid?

Conflict Detection and Reporting

Package Sources

How Sources Work

Source Priority

Project Structure

Usage Examples

Basic Workflow

Dependency Tree Visualization

Resolution Output

Package Information

Advanced Usage

Configuration

Robustness Configuration

Virtual Environments

Export/Import

Repository Management

Use Cases

Commands

Performance

Resolution Performance

Caching Performance

Optimization Techniques

Use Cases

1. Python Projects

2. JavaScript Projects

3. Mixed Projects

4. CI/CD Pipelines

Getting Help

Contributing

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages