Skip to content

Commit 60192cb

Browse files
authored
Revise 'Build and Installation' to 'Build and Dependency Management'
Updated section title and expanded content on build and dependency management for various programming languages.
1 parent 34742f0 commit 60192cb

1 file changed

Lines changed: 136 additions & 4 deletions

File tree

ReplicationTutorial.md

Lines changed: 136 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ This tutorial will cover the basic of putting together a high-quality replicatio
99
## Table of Contents
1010
* [Tools](#tools)
1111
* [Version Control](#version-control)
12-
* [Build and Installation](build-and-installation)
12+
* [Build and Dependendecy Management](#build-and-dependency-management)
1313
* [Environment Management](#environment-management)
1414
* [Scripting](#scripting)
1515
* [Replication Package](#replication-package)
@@ -45,23 +45,155 @@ Then add it as a remote and push (you need to set up an SSH key first).
4545
~/path/to/project> git push -u origin main
4646
```
4747

48-
### Build and Installation
48+
### Build and Dependency Management
4949

5050
Most projects these days are written in high-level languages that do not require compilation. This simplifies things, but it can still be complicated to package your software in a way that makes it easy for another person to install in on their computer from scratch.
5151

5252
#### C/C++
5353

54-
For codes written in C/C++, it is almost always necessary to provide either a hand-generated Makefile or a CMake setup to make building easy. Making C/C++ codes portable is difficult, so ensuring the code builds on Linux is probably the lowest common denominator.
54+
If you have a multi-file project coded natively in C/C++, using a tool to help other build code is a necessity. Below is a quick summary, but more detailed information is [here](CppBuild.md)
55+
56+
* The Big Picture
57+
- **No single standard**: C/C++ has the most fragmented build ecosystem of any major language
58+
- **CMake dominates**: ~60-70% of modern C++ projects use CMake despite its complexity
59+
- **Context matters**: The "best" build system depends heavily on project size, platform needs, and team experience
60+
61+
* Meta-build vs. Direct
62+
- **Meta-build systems** (CMake, Meson, Premake): Generate native build files
63+
- **Direct systems** (Make, Ninja, Bazel): Execute builds themselves
64+
65+
* Quick Recommendations
66+
- **Small/medium projects**: Use **Meson** (simpler) or **CMake** (more ecosystem support)
67+
- **Large monorepos**: Use **Bazel** if you need extreme scale
68+
- **Cross-platform GUI apps**: Use **CMake** for best IDE integration
69+
70+
* Managing Dependencies
71+
- Build systems traditionally only build code
72+
- Dependency management is separate (Conan, vcpkg, system packages)
73+
- Unlike Python/Julia/R, no unified solution for both
74+
- **Emerging trend**: Tools like build2 and xmake integrate both
75+
- Linux
76+
- The native package manager is usually sufficient for installing dependencies.
77+
- Docker can help to easily create an isolated, clean Linux environment.
78+
- Conda with `msys2-bash` is a nice alternative for using Unix tools in Windows/OSX.
79+
- [`linuxbrew'] also provides a way of installing dependencies thst is distribution-independent
80+
- Windows: Conda with `msys2-bash` is a nice alternative for using Unix tools in Windows/OSX.
81+
- OS/X: Homebrew
82+
83+
* When to Use What
84+
85+
| Build System | Best For | Avoid If |
86+
|--------------|----------|----------|
87+
| **CMake** | Cross-platform, ecosystem support, industry standard | You want simple syntax, small hobby projects |
88+
| **Meson** | New projects, fast builds, clean syntax | Need Windows-specific features, established CMake workflow |
89+
| **Make** | Simple Unix projects, understanding legacy code | Cross-platform needs, large projects |
90+
| **Bazel** | Monorepos, massive scale, Google-like setups | Small teams, simple projects, Windows-primary |
91+
| **Autotools** | Traditional Unix distribution | Modern projects, Windows support, maintainability |
92+
| **xmake/Premake** | Game development, Lua enthusiasts | Need mature ecosystem, enterprise support |
5593

5694
#### Python
5795

96+
Highlights of Python's dependency mechanism are here. More details available [here](PythonDeps.md).
5897

98+
* Fragmented Ecosystem
99+
- **Multiple competing standards**: `requirements.txt`, `setup.py`, `pyproject.toml`, Pipfile, etc.
100+
- **No built-in solution**: Unlike Julia, Python requires third-party tools for advanced dependency management
101+
- **Evolution over time**: The ecosystem has gradually moved from `setup.py``pyproject.toml`
102+
103+
* Modern Best Practices
104+
- **pyproject.toml**: Standardized format (PEP 518, 621) for project configuration and dependencies
105+
- **Lock files**: Poetry, Pipenv, or pip-tools provide deterministic, reproducible builds
106+
- **Virtual environments**: `venv` or `virtualenv` for project isolation (separate from dependency specification)
107+
108+
* Tool Landscape
109+
- **pip + requirements.txt**: Simple, ubiquitous, but lacks advanced features
110+
- **Poetry**: All-in-one solution with dependency resolution, virtual env management, and publishing
111+
- **Pipenv**: Combines Pipfile + lock file with virtual environment management
112+
- **pip-tools**: Minimalist approach to generate lock files from requirements.in
113+
114+
* Version Specification
115+
- **Rich syntax**: `==`, `>=`, `~=`, `!=` with multiple constraints
116+
- **Dependency resolution**: Third-party tools handle complex version conflicts
117+
- **Optional dependencies**: `extras_require` or `optional-dependencies` for feature-specific packages
118+
119+
* Key Differences from Other Languages
120+
- **Not built into the language**: Requires external package manager (pip) and tools
121+
- **Separation of concerns**: Virtual environments (isolation) vs. dependency files (specification)
122+
- **Massive ecosystem**: PyPI hosts 500,000+ packages with minimal curation
123+
- **Ongoing standardization**: Community still converging on best practicesPython offers several methods for specifying project dependencies, each suited for different use cases:
59124

60125
#### Julia
61126

127+
Highlights of Python's dependency mechanism are here. More details available [here](JuliaDeps.md).
128+
129+
* Built-in and Unified
130+
- **Native package manager (Pkg)**: Fully integrated into Julia, no external tools needed
131+
- **Single standard**: No fragmentation—everyone uses the same system
132+
- **Batteries included**: Dependency resolution, environments, and versioning work out of the box
133+
134+
* Two-File System
135+
- **Project.toml**: Human-readable file specifying direct dependencies and compatibility
136+
- **Manifest.toml**: Auto-generated lock file with complete dependency graph and exact versions
137+
- **Both committed to version control**: Ensures reproducibility across machines
138+
139+
* Sophisticated Resolution
140+
- **Automatic conflict resolution**: Built-in solver handles complex dependency graphs
141+
- **Semantic versioning**: First-class support with `[compat]` specifications
142+
- **Transitive dependencies**: Automatically resolved and tracked in Manifest.toml
143+
144+
* UUID-Based Identification
145+
- **No naming conflicts**: Packages identified by UUID, not just name
146+
- **Multiple registries**: Can use packages from different sources without collision
147+
- **Global uniqueness**: UUIDs prevent the "namespace problem"
148+
149+
* Environment Management
150+
- **Project-specific environments**: Easy switching with `Pkg.activate()`
151+
- **Isolated by default**: Each project can have different package versions
152+
- **No separate tool needed**: Unlike Python's venv/virtualenv, it's built into Pkg
153+
154+
* Developer Experience
155+
- **Precompilation**: Packages are precompiled for faster loading
156+
- **Development mode**: `Pkg.develop()` for working on packages locally
157+
- **Integrated testing**: `Pkg.test()` runs package tests in isolated environments
158+
159+
* Key Advantages
160+
- **Zero configuration**: Works perfectly without setup or config files
161+
- **Reproducible by default**: Lock files are generated automatically
162+
- **Scientific computing focus**: Designed with reproducibility in mind from day one
163+
- **Fast and efficient**: Modern resolver is quick even with large dependency trees
164+
62165
#### R
63166

64-
### Dependency Management
167+
Highlights of R's dependency mechanism are here. More details available [here](RDeps.md).
168+
169+
* Built-in Tools
170+
- **Simple but limited**: `install.packages()` and `library()` work globally without isolation
171+
- **Automatic dependency installation**: Dependencies listed in `Imports` and `Depends` are installed automatically
172+
- **No sophisticated version resolution**: Uses latest compatible versions from repositories
173+
174+
* Modern Standard: renv
175+
- **Project-specific libraries**: Each project has isolated package installations
176+
- **Reproducible environments**: `renv.lock` captures exact package versions
177+
- **Global package cache**: Saves disk space by sharing packages across projects
178+
- **Simple workflow**: `init()`, `snapshot()`, `restore()` cover most use cases
179+
180+
* Package Repositories
181+
- **CRAN**: Primary repository with ~20,000 packages and strict quality control
182+
- **Bioconductor**: Specialized repository for bioinformatics packages
183+
- **GitHub**: Access to development versions and custom packages
184+
185+
* For Package Development
186+
- **DESCRIPTION file**: Specifies `Imports`, `Suggests`, and `Depends`
187+
- **devtools integration**: Streamlined development workflow
188+
- **Version specifications**: Support for minimum versions and ranges
189+
190+
* Differences from Python/Julia
191+
- Environment isolation was added later (not built-in from the start)
192+
- Less sophisticated dependency resolution compared to Julia's Pkg
193+
- More centralized ecosystem (CRAN curation vs. PyPI's open model)
194+
- **renv** fills the role of Python's Poetry or Julia's built-in Pkg
195+
196+
### Isolated Environments
65197

66198
* [Jupyter Notebooks](https://jupyter.org)
67199
* [Binder](https://mybinder.org)

0 commit comments

Comments
 (0)