Diagnostics

These stages compute and/or plot diagnostics of catalogs or other data

• TXDiagnosticQuantiles - Measure quantiles of various values in the shear catalog.

• TXSourceDiagnosticPlots - Make diagnostic plots of the shear catalog

• TXLensDiagnosticPlots - Make diagnostic plots of the lens catalog

• TXPSFDiagnostics - Make histograms of PSF values

• TXPSFMomentCorr - Compute PSF Moments

• TXTauStatistics - Compute and plot PSF Tau statistics where the definition of Tau stats are eq. 20-22

• TXRoweStatistics - Compute and plot PSF Rowe statistics

• TXGalaxyStarShear - Compute and plot star x galaxy and star x star correlations.

• TXGalaxyStarDensity - Compute and plot star x galaxy and star x star density correlations

• TXBrighterFatterPlot - Compute and plot a diagnostic of the brighter-fatter effect

• TXGammaTFieldCenters - Make diagnostic 2pt measurements of tangential shear around field centers

• TXGammaTStars - Make diagnostic 2pt measurements of tangential shear around stars

• TXGammaTRandoms - Make diagnostic 2pt measurements of tangential shear around randoms

• TXApertureMass - Measure the aperture mass statistics with TreeCorr

• TXFocalPlanePlot - Make diagnostic plot of mean measured ellipticity and residual ellipticity

class txpipe.diagnostics.TXDiagnosticQuantiles(args, comm=None, aliases=None)

Measure quantiles of various values in the shear catalog.

This uses a library called “Distogram” which builds a histogram that it gradually updates, tweaking the edges as it goes along. This means we don’t have to load the full data into memory ever.

The algorithm used can be noisy if there are large outliers in the data, but after selection cuts are made this seems to be okay here, with all the quantiles for the base quantities (T, s2n, T_psf, g1_psf) within 10% of the true quantile and the majority within 1%. For our purposes (defining bins for diagonstics) this is fine.

Inputs:

• shear_catalog: ShearCatalog

• shear_tomography_catalog: TomographyCatalog

Outputs:

• shear_catalog_quantiles: HDFFile

Parallel: Yes - Dask

Configuration

• shear_prefix: (str) Default=mcal_.
• psf_prefix: (str) Default=mcal_psf_.
• nbins: (int) Default=20.
• chunk_rows: (int) Default=0.
• bands: (str) Default=riz.

class txpipe.diagnostics.TXSourceDiagnosticPlots(args, comm=None, aliases=None)

Make diagnostic plots of the shear catalog

This includes both tomographic and 2D measurements, and includes the PSF as a function of various quantities

Inputs:

• shear_catalog: ShearCatalog

• shear_tomography_catalog: TomographyCatalog

• shear_catalog_quantiles: HDFFile

Outputs:

• g_psf_T: PNGFile

• g_psf_g: PNGFile

• g1_hist: PNGFile

• g2_hist: PNGFile

• g_snr: PNGFile

• g_T: PNGFile

• g_colormag: PNGFile

• source_snr_hist: PNGFile

• source_mag_hist: PNGFile

• response_hist: PNGFile

• g_psf_T_out: TextFile

• g_psf_g_out: TextFile

• g_snr_out: TextFile

• g_T_out: TextFile

Parallel: Yes - MPI

Configuration

• chunk_rows: (int) Default=100000.
• delta_gamma: (float) Default=0.02.
• shear_prefix: (str) Default=mcal_.
• psf_prefix: (str) Default=mcal_psf_.
• nbins: (int) Default=20.
• g_min: (float) Default=-0.03.
• g_max: (float) Default=0.05.
• psfT_min: (float) Default=0.04.
• psfT_max: (float) Default=0.36.
• T_min: (float) Default=0.04.
• T_max: (float) Default=4.0.
• s2n_min: (int) Default=10.
• s2n_max: (int) Default=300.
• psf_unit_conv: (bool) Default=False.
• bands: (str) Default=riz.

class txpipe.diagnostics.TXLensDiagnosticPlots(args, comm=None, aliases=None)

Make diagnostic plots of the lens catalog

Currently this consists only of histograms of SNR and mag.

Inputs:

• photometry_catalog: HDFFile

• lens_tomography_catalog: TomographyCatalog

Outputs:

• lens_snr_hist: PNGFile

• lens_mag_hist: PNGFile

Parallel: Yes - Dask

Configuration

• block_size: (int) Default=0.
• delta_gamma: (float) Default=0.02.
• mag_min: (int) Default=18.
• mag_max: (int) Default=28.
• snr_min: (int) Default=5.
• snr_max: (int) Default=200.
• bands: (str) Default=ugrizy.

class txpipe.psf_diagnostics.TXPSFDiagnostics(args, comm=None, aliases=None)

Make histograms of PSF values

This makes histograms of e1 and e2 residuals, and of the fractional PSF size excess.

Inputs:

• star_catalog: HDFFile

Outputs:

• e1_psf_residual_hist: PNGFile

• e2_psf_residual_hist: PNGFile

• T_frac_psf_residual_hist: PNGFile

• star_psf_stats: YamlFile

Parallel: Yes - MPI

Configuration

class txpipe.psf_diagnostics.TXPSFMomentCorr(args, comm=None, aliases=None)

Compute PSF Moments

Inputs:

• star_catalog: HDFFile

Outputs:

• moments_stats: HDFFile

Parallel: No - Serial

Configuration

• min_sep: (float) Default=0.5.
• max_sep: (float) Default=250.0.
• nbins: (int) Default=20.
• bin_slop: (float) Default=0.01.
• sep_units: (str) Default=arcmin.
• subtract_mean: (bool) Default=False.

class txpipe.psf_diagnostics.TXTauStatistics(args, comm=None, aliases=None)

Compute and plot PSF Tau statistics where the definition of Tau stats are eq. 20-22 of Gatti et al 2023.

Inputs:

• binned_shear_catalog: ShearCatalog

• star_catalog: HDFFile

• rowe_stats: HDFFile

Outputs:

• tau0: PNGFile

• tau2: PNGFile

• tau5: PNGFile

• tau_stats: HDFFile

Parallel: No - Serial

Configuration

• min_sep: (float) Default=0.5.
• max_sep: (float) Default=250.0.
• nbins: (int) Default=20.
• bin_slop: (float) Default=0.01.
• sep_units: (str) Default=arcmin.
• npatch: (int) Default=150.
• psf_size_units: (str) Default=sigma.
• subtract_mean: (bool) Default=False.
• dec_cut: (bool) Default=True.
• star_type: (str) Default=PSF-reserved.
• cov_method: (str) Default=bootstrap.
• flip_g2: (bool) Default=False.
• tomographic: (bool) Default=True.

class txpipe.psf_diagnostics.TXRoweStatistics(args, comm=None, aliases=None)

Compute and plot PSF Rowe statistics

People sometimes think that these statistics are called the Rho statistics, because we usually use that letter for them. Not so. They are named after the wonderful Barney Rowe, now sadly lost to high finance, who presented the first two of them in MNRAS 404, 350 (2010).

Inputs:

• star_catalog: HDFFile

• patch_centers: TextFile

Outputs:

• rowe134: PNGFile

• rowe25: PNGFile

• rowe0: PNGFile

• rowe_stats: HDFFile

Parallel: No - Serial

Configuration

• min_sep: (float) Default=0.5.
• max_sep: (float) Default=250.0.
• nbins: (int) Default=20.
• bin_slop: (float) Default=0.01.
• sep_units: (str) Default=arcmin.
• psf_size_units: (str) Default=sigma.
• definition: (str) Default=des-y1.
• subtract_mean: (bool) Default=False.
• star_type: (str) Default=PSF-reserved.
• var_method: (str) Default=bootstrap.
• flip_g2: (bool) Default=False.

class txpipe.psf_diagnostics.TXGalaxyStarShear(args, comm=None, aliases=None)

Compute and plot star x galaxy and star x star correlations.

These are shape correlations; they differ from the Rowe stats slightly because they measure star values not interpolated PSF values.

Inputs:

• shear_catalog: ShearCatalog

• star_catalog: HDFFile

• shear_tomography_catalog: TomographyCatalog

Outputs:

• star_shear_test: PNGFile

• star_star_test: PNGFile

• star_shear_stats: HDFFile

Parallel: No - Serial

Configuration

• min_sep: (float) Default=0.5.
• max_sep: (float) Default=250.0.
• nbins: (int) Default=20.
• bin_slop: (float) Default=0.1.
• sep_units: (str) Default=arcmin.
• psf_size_units: (str) Default=sigma.
• shear_catalog_type: (str) Default=metacal.
• star_type: (str) Default=PSF-reserved.
• flip_g2: (bool) Default=False.

class txpipe.psf_diagnostics.TXGalaxyStarDensity(args, comm=None, aliases=None)

Compute and plot star x galaxy and star x star density correlations

This version uses the source catalog, though a version with lens samples might also be useful.

Inputs:

• shear_catalog: ShearCatalog

• star_catalog: HDFFile

• shear_tomography_catalog: TomographyCatalog

• random_cats: RandomsCatalog

Outputs:

• star_density_test: PNGFile

• star_density_stats: HDFFile

Parallel: No - Serial

Configuration

• min_sep: (float) Default=0.5.
• max_sep: (float) Default=250.0.
• nbins: (int) Default=20.
• bin_slop: (float) Default=0.1.
• sep_units: (str) Default=arcmin.
• psf_size_units: (str) Default=sigma.
• star_type: (str) Default=PSF-reserved.
• flip_g2: (bool) Default=False.

class txpipe.psf_diagnostics.TXBrighterFatterPlot(args, comm=None, aliases=None)

Compute and plot a diagnostic of the brighter-fatter effect

This plots the mean e1, e2, and size difference between that interpolated at star locations and the star values themselves, as a function of magnitude.

Inputs:

• star_catalog: HDFFile

Outputs:

• brighter_fatter_plot: PNGFile

• brighter_fatter_data: HDFFile

Parallel: No - Serial

Configuration

• band: (str) Default=r.
• nbin: (int) Default=20.
• mmin: (float) Default=18.5.
• mmax: (float) Default=23.5.

class txpipe.twopoint_null_tests.TXGammaTFieldCenters(args, comm=None, aliases=None)

Make diagnostic 2pt measurements of tangential shear around field centers

This subclass of the standard TXTwoPoint uses the centers of exposure fields as “lenses”, as a systematics test.

Inputs:

• binned_shear_catalog: ShearCatalog

• shear_photoz_stack: QPNOfZFile

• lens_photoz_stack: QPNOfZFile

• random_cats: RandomsCatalog

• exposures: HDFFile

• patch_centers: TextFile

• tracer_metadata: HDFFile

Outputs:

• gammat_field_center: SACCFile

• gammat_field_center_plot: PNGFile

Parallel: Yes - MPI

Configuration

• calcs: (list) Default=[0, 1, 2].
• min_sep: (float) Default=2.5.
• max_sep: (int) Default=250.
• nbins: (int) Default=20.
• bin_slop: (float) Default=0.1.
• sep_units: (str) Default=arcmin.
• flip_g1: (bool) Default=False.
• flip_g2: (bool) Default=True.
• verbose: (int) Default=1.
• reduce_randoms_size: (float) Default=1.0.
• var_method: (str) Default=shot.
• npatch: (int) Default=5.
• use_true_shear: (bool) Default=False.
• subtract_mean_shear: (bool) Default=False.
• use_randoms: (bool) Default=True.
• patch_dir: (str) Default=./cache/patches.
• low_mem: (bool) Default=False.
• chunk_rows: (int) Default=100000.
• share_patch_files: (bool) Default=False.
• use_subsampled_randoms: (bool) Default=False.

class txpipe.twopoint_null_tests.TXGammaTStars(args, comm=None, aliases=None)

Make diagnostic 2pt measurements of tangential shear around stars

This subclass of the standard TXTwoPoint uses the centers of stars as “lenses”, as a systematics test.

Inputs:

• binned_shear_catalog: ShearCatalog

• shear_tomography_catalog: TomographyCatalog

• shear_photoz_stack: QPNOfZFile

• lens_photoz_stack: QPNOfZFile

• random_cats: RandomsCatalog

• binned_star_catalog: HDFFile

• patch_centers: TextFile

• tracer_metadata: HDFFile

• binned_random_catalog: HDFFile

Outputs:

• gammat_bright_stars: SACCFile

• gammat_bright_stars_plot: PNGFile

• gammat_dim_stars: SACCFile

• gammat_dim_stars_plot: PNGFile

Parallel: Yes - MPI

Configuration

• calcs: (list) Default=[0, 1, 2].
• min_sep: (float) Default=2.5.
• max_sep: (int) Default=100.
• nbins: (int) Default=20.
• bin_slop: (int) Default=1.
• sep_units: (str) Default=arcmin.
• flip_g1: (bool) Default=False.
• flip_g2: (bool) Default=True.
• verbose: (int) Default=1.
• reduce_randoms_size: (float) Default=1.0.
• var_method: (str) Default=shot.
• npatch: (int) Default=5.
• use_true_shear: (bool) Default=False.
• subtract_mean_shear: (bool) Default=False.
• use_randoms: (bool) Default=True.
• patch_dir: (str) Default=./cache/patches.
• low_mem: (bool) Default=False.
• chunk_rows: (int) Default=100000.
• share_patch_files: (bool) Default=False.
• use_subsampled_randoms: (bool) Default=False.

class txpipe.twopoint_null_tests.TXGammaTRandoms(args, comm=None, aliases=None)

Make diagnostic 2pt measurements of tangential shear around randoms

It’s not clear to me that this is a useful null test; if it was we wouldn’t need to subtrac this term in the Landay-Szalay estimator.

This subclass of the standard TXTwoPoint uses the centers of stars as “lenses”, as a systematics test.

Inputs:

• binned_shear_catalog: ShearCatalog

• shear_photoz_stack: QPNOfZFile

• random_cats: RandomsCatalog

• patch_centers: TextFile

• tracer_metadata: HDFFile

Outputs:

• gammat_randoms: SACCFile

• gammat_randoms_plot: PNGFile

Parallel: Yes - MPI

Configuration

• calcs: (list) Default=[0, 1, 2].
• min_sep: (float) Default=2.5.
• max_sep: (int) Default=100.
• nbins: (int) Default=20.
• bin_slop: (int) Default=1.
• sep_units: (str) Default=arcmin.
• flip_g1: (bool) Default=False.
• flip_g2: (bool) Default=True.
• verbose: (int) Default=1.
• reduce_randoms_size: (float) Default=1.0.
• var_method: (str) Default=shot.
• npatch: (int) Default=5.
• use_true_shear: (bool) Default=False.
• subtract_mean_shear: (bool) Default=False.
• use_randoms: (bool) Default=False.
• patch_dir: (str) Default=./cache/patches.
• low_mem: (bool) Default=False.
• chunk_rows: (int) Default=100000.
• share_patch_files: (bool) Default=False.
• use_subsampled_randoms: (bool) Default=False.

class txpipe.twopoint_null_tests.TXApertureMass(args, comm=None, aliases=None)

Measure the aperture mass statistics with TreeCorr

There are real and imaginary components of the aperture mass and its cross term.

Inputs:

• binned_shear_catalog: ShearCatalog

• shear_photoz_stack: QPNOfZFile

• patch_centers: TextFile

• tracer_metadata: HDFFile

Outputs:

• aperture_mass_data: SACCFile

Parallel: Yes - MPI

Configuration

• calcs: (list) Default=[0, 1, 2].
• min_sep: (float) Default=0.5.
• max_sep: (float) Default=300.0.
• nbins: (int) Default=15.
• bin_slop: (float) Default=0.02.
• sep_units: (str) Default=arcmin.
• flip_g1: (bool) Default=False.
• flip_g2: (bool) Default=True.
• verbose: (int) Default=1.
• source_bins: (list) Default=[-1].
• lens_bins: (list) Default=[-1].
• reduce_randoms_size: (float) Default=1.0.
• var_method: (str) Default=jackknife.
• use_true_shear: (bool) Default=False.
• subtract_mean_shear: (bool) Default=False.
• use_randoms: (bool) Default=False.
• low_mem: (bool) Default=False.
• patch_dir: (str) Default=./cache/patches.
• chunk_rows: (int) Default=100000.
• share_patch_files: (bool) Default=False.

class txpipe.spatial_diagnostics.TXFocalPlanePlot(args, comm=None, aliases=None)

Make diagnostic plot of mean measured ellipticity and residual ellipticity as a function of position in the focal plane

Inputs:

• star_catalog: HDFFile

Outputs:

• focalplane_g: PNGFile

Parallel: No - Serial

Configuration