Contents

Incorporating masks into calibrated science images

6.4. Incorporating masks into calibrated science images

There are three ways of determining which pixels in a CCD image may need to be masked (this is in addition to whatever mask or bit fields the observatory at which you are taking images may provide).

Two of them are the same for all of the science images:

The third, identifying cosmic rays, discussed in Cosmic ray removal, will by its nature be different for each science image.

The first two masks could be added to science images at the time the science images are calibrated, if desired. They are added to the science images here, as a separate step, because in many situations it is fine to omit masking entirely and there is no particular advantage to introducing it earlier.

We begin, as usual, with a couple of imports.

from pathlib import Path

from astropy import units as u
from astropy.nddata import CCDData

import ccdproc as ccdp

6.4.1. Read masks that are the same for all of the science images

In previous notebooks we constructed a mask based on the dark current and a mask created by ccdmask from a flat image. Displaying the summary of the the information about the reduced images is a handy way to determine which files are the masks.

ex2_path = Path('example2-reduced')

ifc = ccdp.ImageFileCollection(ex2_path)
ifc.summary['file', 'imagetyp']
Table length=36
fileimagetyp
str31str9
AutoFlat-PANoRot-r-Bin1-001.fitFLAT
AutoFlat-PANoRot-r-Bin1-002.fitFLAT
AutoFlat-PANoRot-r-Bin1-003.fitFLAT
AutoFlat-PANoRot-r-Bin1-004.fitFLAT
AutoFlat-PANoRot-r-Bin1-005.fitFLAT
AutoFlat-PANoRot-r-Bin1-006.fitFLAT
AutoFlat-PANoRot-r-Bin1-007.fitFLAT
AutoFlat-PANoRot-r-Bin1-008.fitFLAT
AutoFlat-PANoRot-r-Bin1-009.fitFLAT
AutoFlat-PANoRot-r-Bin1-010.fitFLAT
......
Dark-S001-R001-C007-NoFilt.fitDARK
Dark-S001-R001-C008-NoFilt.fitDARK
Dark-S001-R001-C009-NoFilt.fitDARK
Dark-S001-R001-C020-NoFilt.fitDARK
combined_bias.fitBIAS
combined_dark_90.000.fitDARK
combined_flat_filter_r.fitFLAT
kelt-16-b-S001-R001-C084-r.fitLIGHT
kelt-16-b-S001-R001-C125-r.fitLIGHT
mask_from_ccdmask.fitsflat mask

We read each of those in below, converting the mask to boolean after we read it.

mask_ccdmask = CCDData.read(ex2_path / 'mask_from_ccdmask.fits', unit=u.dimensionless_unscaled)
mask_ccdmask.data = mask_ccdmask.data.astype('bool')

mask_hot_pix = CCDData.read(ex2_path / 'mask_from_dark_current.fits', unit=u.dimensionless_unscaled)
mask_hot_pix.data = mask_hot_pix.data.astype('bool')

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Input In [3], in <cell line: 4>()
      1 mask_ccdmask = CCDData.read(ex2_path / 'mask_from_ccdmask.fits', unit=u.dimensionless_unscaled)
      2 mask_ccdmask.data = mask_ccdmask.data.astype('bool')
----> 4 mask_hot_pix = CCDData.read(ex2_path / 'mask_from_dark_current.fits', unit=u.dimensionless_unscaled)
      5 mask_hot_pix.data = mask_hot_pix.data.astype('bool')

File /usr/share/miniconda/envs/test/lib/python3.8/site-packages/astropy/nddata/mixins/ndio.py:59, in NDDataRead.__call__(self, *args, **kwargs)
     58 def __call__(self, *args, **kwargs):
---> 59     return self.registry.read(self._cls, *args, **kwargs)

File /usr/share/miniconda/envs/test/lib/python3.8/site-packages/astropy/io/registry/core.py:197, in UnifiedInputRegistry.read(self, cls, format, cache, *args, **kwargs)
    195 try:
    196     ctx = get_readable_fileobj(args[0], encoding='binary', cache=cache)
--> 197     fileobj = ctx.__enter__()
    198 except OSError:
    199     raise

File /usr/share/miniconda/envs/test/lib/python3.8/contextlib.py:113, in _GeneratorContextManager.__enter__(self)
    111 del self.args, self.kwds, self.func
    112 try:
--> 113     return next(self.gen)
    114 except StopIteration:
    115     raise RuntimeError("generator didn't yield") from None

File /usr/share/miniconda/envs/test/lib/python3.8/site-packages/astropy/utils/data.py:271, in get_readable_fileobj(name_or_obj, encoding, cache, show_progress, remote_timeout, sources, http_headers)
    266 if is_url:
    267     name_or_obj = download_file(
    268         name_or_obj, cache=cache, show_progress=show_progress,
    269         timeout=remote_timeout, sources=sources,
    270         http_headers=http_headers)
--> 271 fileobj = io.FileIO(name_or_obj, 'r')
    272 if is_url and not cache:
    273     delete_fds.append(fileobj)

FileNotFoundError: [Errno 2] No such file or directory: 'example2-reduced/mask_from_dark_current.fits'

6.4.1.1. Combining the masks

We combine the masks using a logical “OR” since we want to mask out pixels that are bad for any reason.

combined_mask = mask_ccdmask.data | mask_hot_pix.data

It turns out we are masking roughly 0.056% of the pixels so far.

combined_mask.sum()

6.4.2. Detect cosmic rays

Cosmic ray detection was discussed in detail in an earlier section. Here we loop over all of the calibrated science images and:

  • detect cosmic rays in them,

  • combine the cosmic ray mask with the mask that applies to all images,

  • set the mask of the image to the overall mask, and

  • save the image, overwriting the calibrated science image without the mask.

Since the cosmic ray detection takes a while, a status message is displayed before each image is processed.

ifc.files_filtered()
for ccd, file_name in ifc.ccds(imagetyp='light', return_fname=True):
    print('Working on file {}'.format(file_name))
    new_ccd = ccdp.cosmicray_lacosmic(ccd, readnoise=10, sigclip=8, verbose=True)
    overall_mask = new_ccd.mask | combined_mask
    # If there was already a mask, keep it.
    if ccd.mask is not None:
        ccd.mask = ccd.mask | overall_mask
    else:
        ccd.mask = overall_mask
    # Files can be overwritten only with an explicit option
    ccd.write(ifc.location / file_name, overwrite=True)

previous

6.3. Cosmic ray removal