AstrometricCorrections

class macauff.AstrometricCorrections(psf_fwhms, numtrials, nn_radius, dens_search_radius, save_folder, gal_wavs_micron, gal_ab_offsets, gal_filtnames, gal_alavs, dm, dd_params, l_cut, ax1_mids, ax2_mids, ax_dimension, mag_arrays, mag_slices, sig_slices, n_pool, npy_or_csv, coord_or_chunk, pos_and_err_indices, mag_indices, snr_indices, mag_names, correct_astro_mag_indices_index, coord_system, saturation_magnitudes, pregenerate_cutouts, n_r, n_rho, max_rho, mn_fit_type, trifilepath=None, maglim_f=None, magnum=None, tri_num_faint=None, trifilterset=None, trifiltnames=None, tri_dens_cube=None, tri_dens_array=None, use_photometric_uncertainties=False, cutout_area=None, cutout_height=None, single_sided_auf=True, chunks=None, return_nm=False, apply_proper_motion_flag=False, pm_indices=None, pm_ref_epoch_or_index=None, pm_move_to_epoch=None)

Bases: object

Class to calculate any potential corrections to quoted astrometric precisions in photometric catalogues, based on reliable cross-matching to a well-understood second dataset.

Initialisation of AstrometricCorrections, accepting inputs required for the running of the optimisation and parameterisation of astrometry of a photometric catalogue, benchmarked against a catalogue of much higher astrometric resolution and precision, such as Gaia or the Hubble Source Catalog.

Parameters:

psf_fwhmslist or numpy.ndarray of floats

The full-widths at half-maximum of the Point Spread Functions, used to determine the sizes of the PSF for perturber placement purposes.

numtrialsinteger

Number of simulations to run when deriving pertubation statistics.

nn_radiusfloat

Size of nearest-neighbour search for construction of intermediate cross-match distributions, in arcseconds.

dens_search_radiusfloat

Radius out to which to search around objects internal to a catalogue, to determine the local normalising density for each source, in degrees.

save_folderstring

Absolute or relative filepath of folder into which to store temporary and generated outputs from the fitting process.

gal_wavs_micronlist or numpy.ndarray of floats

Wavelength, in microns, of the chosen filters, for use in simulating galaxy counts.

gal_ab_offsetslist or numpy.ndarray of floats

The offsets between the filter zero points and the AB magnitude offset.

gal_filtnameslist or numpy.ndarray of strings

Names of the filter in the speclite compound naming convention.

gal_alavslist or numpy.ndarray of floats

Differential reddening vector of the given filters.

dmfloat

Bin spacing for magnitude histograms of TRILEGAL simulations.

dd_paramsnumpy.ndarray

Array, of shape (5, X, 2), containing the parameterisation of the skew-normal used to construct background-dominated PSF perturbations.

l_cutnumpy.ndarray

Array of shape (3,) containing the cuts between the different regimes of background-dominated PSF perturbation.

ax1_midsnumpy.ndarray

Array of longitudes (e.g. RA or l) used to center regions used to determine astrometric corrections across the sky. Depending on ax_correction, either the unique values that with ax2_mids form a rectangle, or a unique ax-ax combination with a corresponding ax2_mid.

ax2_midsnumpy.ndarray

Array of latitudes (Dec/b) defining regions for calculating astrometric corrections. Either unique rectangle values to combine with ax1_mids or unique ax1_mids-ax2_mids pairs, one per entry.

ax_dimensioninteger, either 1 or 2

If 1 then ax1_mids and ax2_mids form unique sides of a rectangle when combined in a grid, or if 2 each ax1_mids-ax2_mids combination is a unique ax-ax pairing used as given.

mag_arrayslist of numpy.ndarrays or numpy.ndarray

List of lists of magnitudes in the filter to derive astrometry for, with each element of mag_arrays[i] a list or array of magnitudes at which to take cuts in the corresponding mag_names[i] filter.

mag_sliceslist of numpy.ndarrays or numpy.ndarray

Widths of interval at which to take slices of magnitudes for deriving astrometric properties. Each mag_slices[i] maps elementwise to each mag_array[i], and hence they should be the same shape, with the two lists or arrays meeting len(mag_arrays) == len(mag_slices).

sig_sliceslist of numpy.ndarrays or numpy.ndarray

Interval widths of quoted astrometric uncertainty to use when isolating individual sets of objects for AUF derivation. Length should match mag_array. List must have the same number of elements as mag_arrays, with each list-element agreeing in length as well; if a numpy array, then np.all(mag_arrays.shape == sig_slices.shape) must be true.

n_poolinteger

The maximum number of threads to use when calling multiprocessing.

npy_or_csvstring, either npy or csv

Indicator as to whether the small chunks of sky to be loaded for each sightline’s evaluation are in binary numpy format or saved to disk as a comma-separated values file.

coord_or_chunkstring, either coord or chunk

String indicating whether intermediate files should be saved with filenames that are unique by two coordinates (l/b or RA/Dec) or some kind of singular “chunk” number. Output filenames would then need to follow 'file_{}{}' or 'file_{}' formatting respectively.

pos_and_err_indiceslist of list or numpy.ndarray of integers

In order, the indices within catalogue “b” and then “a” respecetively of either the .npy or .csv file of the longitudinal (e.g. RA or l), latitudinal (Dec or b), and singular, circular astrometric precision array. Coordinates should be in degrees while precision should be in the same units as sig_slice and those of the nearest-neighbour distances, likely arcseconds. Note that coordinates are reversed between calogues: for example, [[6, 3, 0], [0, 1, 2]] is the case where catalogue “a” has its coordinates in the first three columns in RA/Dec/Err order, while catalogue “b” has its coordinates in a more random order. If use_photometric_uncertainties is True then the third index of the first triplet (i.e., the index for the to-be-fit catalogue’s uncertainties) should be that of a photometric uncertainty rather than an astrometric uncertainty.

mag_indiceslist or numpy.ndarray

In appropriate order, as expected by e.g. CrossMatch inputs and make_perturb_aufs, list the indexes of each magnitude column within either the .npy or .csv file loaded for each sub-catalogue sightline. These should be zero-indexed.

snr_indiceslist or numpy.ndarray

For each mag_indices entry, the corresponding signal-to-noise ratio index in the catalogue.

mag_nameslist or numpy.ndarray of strings

Names of each mag_indices magnitude.

correct_astro_mag_indices_indexinteger

Index into mag_indices of the preferred magnitude to use to construct astrometric scaling relations from. Should generally be the one with the most coverage across all detections in a catalogue, or the one with the most precise fluxes.

coord_systemstring, “equatorial” or “galactic”

Identifier of which coordinate system the data are in. Both datasets must be in the same system, which can either be RA/Dec (equatorial) or l/b (galactic) coordinates.

saturation_magnitudeslist or numpy.ndarray

List of magnitudes brighter than which the given survey suffers saturation, each element matching the filter of mag_indices.

pregenerate_cutoutsboolean or None

Indicates whether sightline catalogues must have been pre-made, or whether they can be generated by AstrometricCorrections using specified lines-of-sight and cutout areas and heights. If None then in-memory catalogues must be passed in the call signature of AstrometricCorrections.

n_rinteger

Number of elements to generate for real-space evaluation of Bessel functions, used in the convolution of AUF PDfs.

n_rhointeger

Number of elements to generate for fourier-space Bessel function evaluation, used in the convolution of AUF PDfs.

max_rhofloat

Largest fourier-space value to integrate functions to during the convolution of AUF PDFs.

mn_fit_typestring

Determines whether we perform quadratic or linear scaling for hyper-parameter fits to data-driven vs quoted astrometric uncertainties. Must either be “quadratic” or “linear.”

trifilepathstring, optional

Filepath of the location into which to save TRILEGAL simulations. If provided tri_dens_cube and tri_dens_array must be None, and maglim_fs, magnums, tri_num_faints, trifilterset, and trifiltnames must be given. Must contain two format {} options in string, for unique ax1-ax2 sightline combination downloads.

maglim_ffloat, optional

Magnitude in the magnum filter down to which sources should be drawn for the “faint” sample. Should be None if tri_dens_cube and tri_dens_array are provided.

magnumfloat, optional

Zero-indexed column number of the chosen filter’s limiting magnitude. Should be None if tri_dens_cube and tri_dens_array are provided.

tri_num_faintinteger, optional

Approximate number of objects to simulate in the chosen filter for TRILEGAL simulations. Should be None if tri_dens_cube and tri_dens_array are provided.

trifiltersetstring, optional

Name of the TRILEGAL filter set for which to generate simulations. Should be None if tri_dens_cube and tri_dens_array are provided.

trifiltnameslist of string, optional

Name of the specific filters to generate perturbation AUF component in. Should be None if tri_dens_cube and tri_dens_array are provided.

tri_dens_cubenumpy.ndarray or None, optional

If given, array of differential source densities, per square degree per magnitude, along with magnitude bins and intervals, for a set of given filters and sky positions, as computed by make_tri_counts. Must be provided if trifilepath is None, else must itself be None.

tri_dens_arraynumpy.ndarray or None, optional

Corresponding sky-coordinate array to extract the relevant column from tri_dens_cube. Must be given if tri_dens_cub is provided, else None.

use_photometric_uncertaintiesboolean, optional

Flag for whether or not to use the photometric uncertainties instead of astrometric uncertainties when deriving astrometric uncertainties as a function of input precision. Defaults to False (use astrometric uncertainties).

cutout_areafloat, optional

The size, in square degrees, of the regions used to simulate AUFs and determine astrometric corrections. Required if pregenerate_cutouts is False.

cutout_heightfloat, optional

The latitudinal height of the rectangular regions used in calculating astrometric corrections. Required if pregenerate_cutouts is False.

single_sided_aufboolean, optional

Flag indicating whether the AUF of catalogue “a” can be ignored when considering match statistics, or if astrometric corrections are being constructed from matches to a catalogue that also suffers significant non-noise-based astrometric uncertainty.

chunks = list or numpy.ndarray of strings, optional

List of IDs for each unique set of data if coord_or_chunk is chunk. In this case, ax_dimension must be 2 and each chunk must correspond to its ax1_mids-ax2_mids coordinate.

return_nmboolean, optional

Flag for whether the output correction arrays m and n should be saved to disk (False) or returned by the function (True).

apply_proper_motion_flagboolean, optional

Flag to indicate if either dataset has proper motions that must be applied to its positions before determining astrometric corrections. Defaults to False, in which case neither dataset will check for, or load, proper motion-based data.

pm_indiceslist of list or numpy.ndarray of integers, optional

If given, must contain a list of two elements, each of which contains the two orthogonal sky-axis proper motions’ indices for the given dataset, in the same reference frame as the positions to be loaded along with positions, SNRs, photometry, as necessary. As this is on a per-catalogue basis, if either (or both) dataset has no motion information, None must be given, with the first element of the list being, to be consistent with pos_and_err_indices, the “b”, to-be-fit, catalogue, with the second element the “a”, “truth” dataset. We could have e.g. [None, [4, 5]] for the case where we apply motion information to our truth catalogue, with proper motion information being stored in the 5th and 6th columns.

pm_ref_epoch_or_indexlist of strings or integers, optional

If provided, necessary when pm_indices is passed, each element, consistent with pm_indices catalogue ordering, must either be a single string, valid for loading into astropy’s Time function, representing a common date of observation for all sources in the file or a single integer, loading from the dataset the astropy Time-valid strings for each object individually as a column of the input file. Again, if no motions are to be applied for an individual catalogue but apply_proper_motion_flag is True, then None must be given for that list element; for the above example we could either supply [None, 6] or [None, 'J2000'] to load per-source epochs from our file or supply a fixed observation date respectively.

pm_move_to_epochstring, optional

If pm_indices is provided this must be given, containing a single, astropy Time valid, string, the final epoch to apply proper motions of all objects to. This must be a single string, rather than a per-catalogue list, to ensure that a common epoch is used when proper motions are to be applied to both datasets.

Methods Summary

__call__([a_cat, b_cat, a_cat_name, ...])	Call function for the correction calculation process.
calc_single_joint_auf(o, i, bins, y, q, num, snr)	Calculates the negative-log-likelihood of a single magnitude slice of match separations as fit with an AUF.
calculate_local_densities_and_nearest_neighbours()	Calculate local normalising catalogue densities and catalogue-catalogue nearest neighbour match pairings for each cutout region.
create_auf_pdfs([min_hist_cut])	Using perturbation AUF simulations, generate probability density functions of perturbation distance for all cutout regions, as well as recording key statistics such as average magnitude or SNR.
finalise_summary_plots()	After running all of the sightlines' fits, generate final summary plots of the sig-sig relations, quality of fits, and resulting "m" and "n" scaling parameters.
fit_auf_joint(p)	Fits all magnitude slices for a single sightline, modelling empirically derived centroid uncertainties in all cases in addition to fixed uncertainty contributions.
fit_uncertainty()	For all magnitudes for eachsightline, fit for the empirical centroid uncertainty relationship that describes the distribution of match separations.
load_catalogue(cat_type, sub_cat_id)	Load specific sightline's catalogue, accounting for catalogue "a" vs "b", filetype, and method by which sightlines are divided.
make_ax_coords([check_b_only])	Derive the unique ax1-ax2 combinations used in fitting astrometry, and calculate corner coordinates based on the size of the box and its central coordinates.
make_catalogue_cutouts()	Generate cutout catalogues for regions as defined by corner ax1-ax2 coordinates.
make_gridspec(name, y, x, ratio, z, **kwargs)	Convenience function to generate a matplotlib canvas and populate it with a gridspec grid.
make_star_galaxy_counts()	Generate differential source counts for each cutout region, simulating both stars and galaxies.
plot_fits_calculate_cdfs()	Calculate poisson CDFs and create verification plots showing the quality of the fits.
plot_snr_mag_sig()	Generate 2-D histograms of SNR, quoted/fit astrometric uncertainty, and photometric magnitude.
plot_star_galaxy_counts()	Plotting routine to display data and model differential source counts, for verification purposes.
simulate_aufs()	Simulate unresolved blended contaminants for each magnitude-sightline combination, for both aperture photometry and background-dominated PSF algorithms.

Methods Documentation

__call__(a_cat=None, b_cat=None, a_cat_name=None, b_cat_name=None, a_cat_func=None, b_cat_func=None, tri_download=True, overwrite_all_sightlines=False, make_plots=False, make_summary_plot=True, seeing_ranges=None, single_or_repeat='single', repeat_unique_visits_list=None)

Call function for the correction calculation process.

Parameters:

a_catnumpy.ndarray or list of numpy.ndarray, optional

A pre-loaded, or list of pre-loaded, reference catalogue for the aiding in the determination the astrometric uncertainties for b_cat. Should be provided if pregenerate_cutouts is None, and must be itself None if a_cat_name is given.

b_catnumpy.ndarray or list of numpy.ndarray, optional

A pre-loaded, or list of pre-loaded, catalogue for which to determination the astrometric uncertainties. Should be provided if pregenerate_cutouts is None, and must be itself None if a_cat_name is given.

a_cat_namestring, optional

Name of the catalogue “a” filename, pre-generated or saved by a_cat_func. Must accept one or two formats via Python string formatting (e.g. 'a_string_{}') that represent chunk, or ax1_mid and ax2_mid, depending on coord_or_chunk. Must be given if a_cat is None.

b_cat_namestring, optional

Name of the catalogue “b” filename. Must accept the same string formatting as a_cat_name. Must be given if b_cat is None.

a_cat_funccallable, optional

Function used to generate reduced catalogue table for catalogue “a”. Must be given if pregenerate_cutouts is False.

b_cat_funccallable

Function used to generate reduced catalogue table for catalogue “b”. Must be given if pregenerate_cutouts is False.

tri_downloadboolean or None, optional

Flag determining if TRILEGAL simulations should be re-downloaded if they already exist on disk. Should be None if tri_dens_cube was given in initialisation.

overwrite_all_sightlinesboolean, optional

Flag for whether to create a totally fresh run of astrometric corrections, regardless of whether``mn_sigs_array`` is saved on disk. Defaults to False.

make_plotsboolean, optional

Determines if intermediate figures are generated in the process of deriving astrometric corrections.

make_summary_plotboolean, optional

If True then final summary plot is created, even if make_plots is False and intermediate plots are not created.

seeing_rangeslist or numpy.ndarray, optional

If make_plots is True, must be a valid 1-, 2-, or 3-length list of floats, the seeing, in arcseconds, of observations, for plotting over astrometric precision vs SNR distributions.

single_or_repeatstring, either “single” or “repeat”

Flag indicating whether catalogue is made of a single set of observations or repeated “visits” to this patch of sky. If “single” then each row is assumed to be unique, but if “repeat” is given then repeat_unique_visits_list must be provided, giving the indices for which row comes from which unique repeat-observation of the sky region.

repeat_unique_visits_listlist of list or numpy.ndarray of integers

If single_or_repeat is “repeat”, then these arrays must be provided, giving the ID of each sub-visit that has been merged into a single catalogue, for use in separating individual visits out for cross-matches and local normalising density calculations. Length must match a_cat and b_cat, one set of visit indices per field pointing.

calc_single_joint_auf(o, i, bins, y, q, num, snr)

Calculates the negative-log-likelihood of a single magnitude slice of match separations as fit with an AUF.

Parameters:

ofloat

The Gaussian uncertainty of the centroid component of the AUF.

iint

Index to access particular magnitude slice.

binsnumpy.ndarray

Array of floats, bin edges of histogram of cross-match distances.

ynumpy.ndarray

Array of floats, counts in each bin of bins of cross-match distances.

qnumpy.ndarray

Array of booleans, flags for whether to use each y bin or not.

numint

Total number of counts within each PDF in y, to convert back to raw counts for Poisson counting statistics.

snr: numpy.ndarray

Array of signal-to-noise ratios, for weighting between different algorithms for the Perturbation component of the AUF.

Returns:

neg_log_like_partnumpy.ndarray

The negative-log-likelihood of dataset of separations given the AUF model.

calculate_local_densities_and_nearest_neighbours()

Calculate local normalising catalogue densities and catalogue-catalogue nearest neighbour match pairings for each cutout region.

create_auf_pdfs(min_hist_cut=100)

Using perturbation AUF simulations, generate probability density functions of perturbation distance for all cutout regions, as well as recording key statistics such as average magnitude or SNR.

Parameters:

min_hist_cutinteger, optional

Number of data points in each magnitude-uncertainty slice to be considered for fitting for scaling relations.

finalise_summary_plots()

After running all of the sightlines’ fits, generate final summary plots of the sig-sig relations, quality of fits, and resulting “m” and “n” scaling parameters.

fit_auf_joint(p)

Fits all magnitude slices for a single sightline, modelling empirically derived centroid uncertainties in all cases in addition to fixed uncertainty contributions.

Parameters:

plist

List containing the scaling values to be fit for.

Returns:

neg_log_likefloat

Negative-log-likelihood of the fit, to be minimised in the wrapping function call.

fit_uncertainty()

For all magnitudes for eachsightline, fit for the empirical centroid uncertainty relationship that describes the distribution of match separations.

Returns:

mfloat

The multiplicative factor for converting old uncertainties to new astrometric precision.

nfloat

Quadratically added constant uncertainty to add to scaling between input and output uncertainties.

load_catalogue(cat_type, sub_cat_id)

Load specific sightline’s catalogue, accounting for catalogue “a” vs “b”, filetype, and method by which sightlines are divided.

Parameters:

cat_typestring, “a” or “b”

Identifier for which of the two catalogues to load.

sub_cat_idlist

Contains the variables to format the name of the catalogue with, in the sense of string.format(x, y, ...). Should either contain ax1 and ax2, or chunk ID, depending on coord_or_chunk.

Returns:

xnumpy.ndarray

The catalogue’s dataset.

make_ax_coords(check_b_only=False)

Derive the unique ax1-ax2 combinations used in fitting astrometry, and calculate corner coordinates based on the size of the box and its central coordinates.

Parameters:

check_b_onlyboolean, optional

Overloadable flag for cases where we can ignore catalogue ‘a’ and only need to check for catalogue ‘b’ existing if pregenerate_cutouts is True.

make_catalogue_cutouts()

Generate cutout catalogues for regions as defined by corner ax1-ax2 coordinates.

make_gridspec(name, y, x, ratio, z, **kwargs)

Convenience function to generate a matplotlib canvas and populate it with a gridspec grid.

Parameters:

namestring

Unique identifier for the figure returned.

yint

Number of rows in the resulting grid of axes.

xint

Number of columns in the figure grid.

ratiofloat

Aspect ratio of axis, with 0.5 being half as tall as it is wide.

zfloat

Width of a single axis in real figure units, likely inches.

Returns:

gs~`matplotlib.gridspec.GridSpec`

The created gridspec instance.

make_star_galaxy_counts()

Generate differential source counts for each cutout region, simulating both stars and galaxies.

plot_fits_calculate_cdfs()

Calculate poisson CDFs and create verification plots showing the quality of the fits.

plot_snr_mag_sig()

Generate 2-D histograms of SNR, quoted/fit astrometric uncertainty, and photometric magnitude.

plot_star_galaxy_counts()

Plotting routine to display data and model differential source counts, for verification purposes.

simulate_aufs()

Simulate unresolved blended contaminants for each magnitude-sightline combination, for both aperture photometry and background-dominated PSF algorithms.