Installation

Package Requirements

Currently there are no strict criteria for installation; it is suggested that you use the most up-to-date package versions available. The one exception there is that the minimum version of Python is set to 3.8, and development is currently focused on Python 3.9+.

The current package requirements, and citations to give for use of macauff in your work where available, are:

  • numpy – Harris et al. 2020 (Nature, 585, 357)

  • scipy – Virtanen et al. 2020 (Nature Methods, 17, 261)

  • astropy – Astropy Collaboration et al. 2013 (A&A, 558, 33); Astropy Collaboration et al. 2018 (ApJ, 156, 123); Astropy Collaboration et al. 2022 (ApJ, 935, 167)

  • matplotlib – Hunter 2007 (Computing in Science & Engineering, 9, 90)

  • skypy – Amara et al. 2021 (JOSS, 6, 3056)

  • speclite – Kirkby et al. 2023 (doi.org/10.5281/zenodo.7734526)

  • pandas – The pandas development team 2020 (doi.org/10.5281/zenodo.3509134)

  • dustmaps – Schlafly & Finkbeiner 2011 (ApJ, 737, 103); Schlegel, Finkbeiner & Davis 1998 (ApJ, 500, 525; doi 10.7910/DVN/EWCNL5); Green 2018 (JOSS, 3, 695)

  • scikit-build-core – Schreiner et al. 2022 (doi.org/10.25080/majora-212e5952-033)

  • cmake

with an optional dependency of

  • mpi4py – Dalcin, Paz, & Storti 2005 (Journal of Parallel and Distributed Computing, 65, 1108); Dalcin & Fang 2021 (Computing in Science & Engineering, 23, 47).

For running the test suite the requirements are:

  • tox

  • pytest

  • sphinx-fortran

  • sphinx-astropy

  • pytest-astropy

  • pytest-cov

  • jupyter – Kluyver et al. 2016 (IOS Press, 87, doi.org/10.3233/978-1-61499-649-1-87)

  • isort

  • pylint

  • pre-commmit.

Additionally, you will need the following to install macauff:

  • gfortran – at least version 8

  • git

Installing the Package

As of now, the main way to install this package is by downloading it from the GitHub repository. We recommend using an Anaconda Distribution, or miniconda, to maintain specific, independent Python installations without the need for root permissions. However, you can also pip install macauff, but note that this version may be behind the latest development version available through GitHub.

Once you have installed your choice of conda, then you can create an initial conda environment:

conda create -n your_environment_name -c conda-forge python=3.9 numpy scipy astropy matplotlib skypy speclite pandas dustmaps scikit-build-core cmake

although you can drop the =3.9, or chose another (later) Python version – remembering the minimum version is 3.8 – if you desire to do so. Then activate this as our Python environment:

conda activate your_environment_name

If you require the additional test packages listed above, for running tests, you can install them separately with:

conda install -c conda-forge tox pytest sphinx-astropy pytest-astropy pytest-cov jupyter isort pylint pre_commit
conda install -c vacumm -c conda-forge sphinx-fortran

To build the jupyter notebooks you will require Pandoc. You will also need to install gfortran in order to compile the fortran code in this package. Instructions for how to install this for Windows, MacOS, or Linux can be found here. Finally, install git if you do not have it on your computer; instructions for installing it on your operating system are available.

Once you have the required packages installed – whether in a new conda environment or otherwise – you can clone the repository:

git clone git://github.com/onoddil/macauff.git

which will place the repository in the folder from which you invoked the git command. Now, from inside the folder that was just created (cd macauff or equivalent), you can either run:

pip install .

or:

pip install . --config-settings=cmake.build-type="Debug"

which will install macauff such that you can import macauff from other folders on your computer. However, if this is to develop the software, your changes will not be reflected in the installed version of the code (and you must re-install using the above command); if you wish to have Python code changes immediately reflected in your pip-installed version of the software, you can install macauff using:

pip install -e .

or:

pip install -e . --config-settings=cmake.build-type="Debug"

where -e is the “editable” flag. Note that the config-settings parameter passed through pip controls the optimisation flag for the compilation of Fortran code. With debug set, macauff will install with -O0 optimisation and numerous additional check/debug flags, while by default the software will be installed with compiler optimisation.

To confirm your installation was successful, you can import macauff in a Python terminal.

OpenMP Setup

Within the Fortran codebase, OpenMP is used in numerous places to parallelise independent operations. For these to work, several additional environment variables must be set.

First, OMP_NUM_THREADS must be set; in bash for example this would be export OMP_NUM_THREADS=10. This should be set as high as reasonably possible to decrease overall runtime as much as possible.

Second, if large matches are likely to be used in the future, GOMP_STACKSIZE (and possibly OMP_STACKSIZE, compiler dependent) should be increased. This should again be set as high as reasonably possible to avoid segmentation faults due to stack size.

Finally, again in the case of large catalogue runs, ulimit -s should be set to a higher number (in the limit ulimit -s unlimited can be used to remove the cap entirely) to avoid issues with overall stacksize.

Testing

To run the main unit test suite, assuming you installed it during the above process, you can run:

tox -e test

If you wish to locally build the documentation – mostly likely if you are improving or extending the documentation, as the docs are available online – you can run:

tox -e build_docs

To match the github actions pre-commit workflow, locally you can run:

SKIP=check-lincc-frameworks-template-version,no-commit-to-branch,check-added-large-files,pytest-check,sphinx-build,pylint pre-commit run --show-diff-on-failure --color=always --all-files

which will run isort and report any issues with the formatting prior to code being merged into the main codebase.

Additionally, pylint can be invoked directly using:

pylint -rn -sn --recursive=y ./src --rcfile=./src/.pylintrc
pylint -rn -sn --recursive=y ./tests --rcfile=./tests/.pylintrc
pylint -rn -sn --recursive=y ./benchmarks --rcfile=./tests/.pylintrc

to run the linter on each folder, with changes being manually made to comply with the reporting.

Getting Started

Once you have installed the package, check out the Quick Start page.