Calibrate dark images
Contents
3.4. Calibrate dark images#
Dark images, like any other images, need to be calibrated. Depending on the data you have and the choices you have made in reducing your data, the steps to reducing your images may include:
Subtracting overscan (only if you decide to subtract overscan from all images).
Trim the image (if it has overscan, whether you are using the overscan or not).
Subtract bias (if you need to scale the calibrated dark frames to a different exposure time).
from pathlib import Path
from astropy.nddata import CCDData
from ccdproc import ImageFileCollection
import ccdproc as ccdp
3.4.1. Example 1: Overscan subtracted, bias not removed#
3.4.1.1. Take a look at what images you have#
First we gather up some information about the raw images and the reduced images
up to this point. These examples have darks stored in a subdirectory of the
folder with the rest of the images, so we create an ImageFileCollection for
each.
ex1_path_raw = Path('example-cryo-LFC')
ex1_images_raw = ImageFileCollection(ex1_path_raw)
ex1_darks_raw = ImageFileCollection(ex1_path_raw / 'darks')
ex1_path_reduced = Path('example1-reduced')
ex1_images_reduced = ImageFileCollection(ex1_path_reduced)
3.4.1.1.1. Raw images, everything except the darks#
ex1_images_raw.summary['file', 'imagetyp', 'exptime', 'filter']
| file | imagetyp | exptime | filter |
|---|---|---|---|
| str14 | str9 | float64 | str2 |
| ccd.001.0.fits | BIAS | 0.0 | i' |
| ccd.002.0.fits | BIAS | 0.0 | i' |
| ccd.003.0.fits | BIAS | 0.0 | i' |
| ccd.004.0.fits | BIAS | 0.0 | i' |
| ccd.005.0.fits | BIAS | 0.0 | i' |
| ccd.006.0.fits | BIAS | 0.0 | i' |
| ccd.014.0.fits | FLATFIELD | 70.001 | g' |
| ccd.015.0.fits | FLATFIELD | 70.011 | g' |
| ccd.016.0.fits | FLATFIELD | 70.001 | g' |
| ccd.017.0.fits | FLATFIELD | 7.0 | i' |
| ccd.018.0.fits | FLATFIELD | 7.0 | i' |
| ccd.019.0.fits | FLATFIELD | 7.0 | i' |
| ccd.037.0.fits | OBJECT | 300.062 | g' |
| ccd.043.0.fits | OBJECT | 300.014 | i' |
3.4.1.1.2. Raw dark frames#
ex1_darks_raw.summary['file', 'imagetyp', 'exptime', 'filter']
| file | imagetyp | exptime | filter |
|---|---|---|---|
| str14 | str4 | float64 | str2 |
| ccd.002.0.fits | BIAS | 0.0 | r' |
| ccd.013.0.fits | DARK | 300.0 | r' |
| ccd.014.0.fits | DARK | 300.0 | r' |
| ccd.015.0.fits | DARK | 300.0 | r' |
| ccd.017.0.fits | DARK | 70.0 | r' |
| ccd.018.0.fits | DARK | 70.0 | r' |
| ccd.019.0.fits | DARK | 70.0 | r' |
| ccd.023.0.fits | DARK | 7.0 | r' |
| ccd.024.0.fits | DARK | 7.0 | r' |
| ccd.025.0.fits | DARK | 7.0 | r' |
3.4.1.2. Decide which calibration steps to take#
This example is, again, one of the chips of the LFC camera at Palomar. In earlier notebooks we have seen that the chip has a useful overscan region, has little dark current except for some hot pixels, and sensor glow in one corner of the chip.
Looking at the list of non-dark images (i.e., the flat and light images) shows that for each exposure time in the non-dark images there is a set of dark exposures that has a matching, or very close to matching, exposure time.
To be more explicit, there are flats with exposure times of 7.0 sec and 70.011 sec and darks with exposure time of 7.0 and 70.0 sec. The dark and flat exposure times are close enough that there is no need to scale them. The two images of an object are each roughly 300 sec, matching the darks with exposure time 300 sec. The very small difference in exposure time, under 0.1 sec, does not need to be compensated for.
Given this, we will:
Subtract overscan from each of the darks. The useful overscan region is XXX (see LINK).
Trim the overscan out of the dark images.
We will not subtract bias from these images because we will not need to rescale them to a different exposure time.
3.4.1.3. Calibrate the individual dark frames#
for ccd, file_name in ex1_darks_raw.ccds(imagetyp='DARK', # Just get the dark frames
ccd_kwargs={'unit': 'adu'}, # CCDData requires a unit for the image if
# it is not in the header
return_fname=True # Provide the file name too.
):
# Subtract the overscan
ccd = ccdp.subtract_overscan(ccd, overscan=ccd[:, 2055:], median=True)
# Trim the overscan
ccd = ccdp.trim_image(ccd[:, :2048])
# Save the result
ccd.write(ex1_path_reduced / file_name)
WARNING: FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 57460.000000 from DATE-OBS'. [astropy.wcs.wcs]
WARNING:astropy:FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 57460.000000 from DATE-OBS'.
WARNING: FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 57461.000000 from DATE-OBS'. [astropy.wcs.wcs]
WARNING:astropy:FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 57461.000000 from DATE-OBS'.
3.4.1.3.1. Reduced images (so far)#
ex1_images_reduced.refresh()
ex1_images_reduced.summary['file', 'imagetyp', 'exptime', 'filter', 'combined']
| file | imagetyp | exptime | filter | combined |
|---|---|---|---|---|
| str17 | str4 | float64 | str2 | object |
| ccd.001.0.fits | BIAS | 0.0 | i' | -- |
| ccd.002.0.fits | BIAS | 0.0 | i' | -- |
| ccd.003.0.fits | BIAS | 0.0 | i' | -- |
| ccd.004.0.fits | BIAS | 0.0 | i' | -- |
| ccd.005.0.fits | BIAS | 0.0 | i' | -- |
| ccd.006.0.fits | BIAS | 0.0 | i' | -- |
| ccd.013.0.fits | DARK | 300.0 | r' | -- |
| ccd.014.0.fits | DARK | 300.0 | r' | -- |
| ccd.015.0.fits | DARK | 300.0 | r' | -- |
| ccd.017.0.fits | DARK | 70.0 | r' | -- |
| ccd.018.0.fits | DARK | 70.0 | r' | -- |
| ccd.019.0.fits | DARK | 70.0 | r' | -- |
| ccd.023.0.fits | DARK | 7.0 | r' | -- |
| ccd.024.0.fits | DARK | 7.0 | r' | -- |
| ccd.025.0.fits | DARK | 7.0 | r' | -- |
| combined_bias.fit | BIAS | 0.0 | i' | True |
3.4.2. Example 2: Overscan not subtracted, bias is removed#
ex2_path_raw = Path('example-thermo-electric')
ex2_images_raw = ImageFileCollection(ex2_path_raw)
ex2_path_reduced = Path('example2-reduced')
ex2_images_reduced = ImageFileCollection(ex2_path_reduced)
We begin by looking at what exposure times we have in this data.
ex2_images_raw.summary['file', 'imagetyp', 'exposure'].show_in_notebook()
WARNING: AstropyDeprecationWarning: show_in_notebook() is deprecated as of 6.1 and to create
interactive tables it is recommended to use dedicated tools like:
- https://github.com/bloomberg/ipydatagrid
- https://docs.bokeh.org/en/latest/docs/user_guide/interaction/widgets.html#datatable
- https://dash.plotly.com/datatable [warnings]
WARNING:astropy:AstropyDeprecationWarning: show_in_notebook() is deprecated as of 6.1 and to create
interactive tables it is recommended to use dedicated tools like:
- https://github.com/bloomberg/ipydatagrid
- https://docs.bokeh.org/en/latest/docs/user_guide/interaction/widgets.html#datatable
- https://dash.plotly.com/datatable
| idx | file | imagetyp | exposure |
|---|---|---|---|
| 0 | AutoFlat-PANoRot-r-Bin1-001.fit | FLAT | 1.0 |
| 1 | AutoFlat-PANoRot-r-Bin1-002.fit | FLAT | 1.0 |
| 2 | AutoFlat-PANoRot-r-Bin1-003.fit | FLAT | 1.0 |
| 3 | AutoFlat-PANoRot-r-Bin1-004.fit | FLAT | 1.0 |
| 4 | AutoFlat-PANoRot-r-Bin1-005.fit | FLAT | 1.0 |
| 5 | AutoFlat-PANoRot-r-Bin1-006.fit | FLAT | 1.02 |
| 6 | AutoFlat-PANoRot-r-Bin1-007.fit | FLAT | 1.06 |
| 7 | AutoFlat-PANoRot-r-Bin1-008.fit | FLAT | 1.11 |
| 8 | AutoFlat-PANoRot-r-Bin1-009.fit | FLAT | 1.16 |
| 9 | AutoFlat-PANoRot-r-Bin1-010.fit | FLAT | 1.21 |
| 10 | Bias-S001-R001-C001-NoFilt.fit | BIAS | 0.0 |
| 11 | Bias-S001-R001-C002-NoFilt.fit | BIAS | 0.0 |
| 12 | Bias-S001-R001-C003-NoFilt.fit | BIAS | 0.0 |
| 13 | Bias-S001-R001-C004-NoFilt.fit | BIAS | 0.0 |
| 14 | Bias-S001-R001-C005-NoFilt.fit | BIAS | 0.0 |
| 15 | Bias-S001-R001-C006-NoFilt.fit | BIAS | 0.0 |
| 16 | Bias-S001-R001-C007-NoFilt.fit | BIAS | 0.0 |
| 17 | Bias-S001-R001-C008-NoFilt.fit | BIAS | 0.0 |
| 18 | Bias-S001-R001-C009-NoFilt.fit | BIAS | 0.0 |
| 19 | Bias-S001-R001-C020-NoFilt.fit | BIAS | 0.0 |
| 20 | Dark-S001-R001-C001-NoFilt.fit | DARK | 90.0 |
| 21 | Dark-S001-R001-C002-NoFilt.fit | DARK | 90.0 |
| 22 | Dark-S001-R001-C003-NoFilt.fit | DARK | 90.0 |
| 23 | Dark-S001-R001-C004-NoFilt.fit | DARK | 90.0 |
| 24 | Dark-S001-R001-C005-NoFilt.fit | DARK | 90.0 |
| 25 | Dark-S001-R001-C006-NoFilt.fit | DARK | 90.0 |
| 26 | Dark-S001-R001-C007-NoFilt.fit | DARK | 90.0 |
| 27 | Dark-S001-R001-C008-NoFilt.fit | DARK | 90.0 |
| 28 | Dark-S001-R001-C009-NoFilt.fit | DARK | 90.0 |
| 29 | Dark-S001-R001-C020-NoFilt.fit | DARK | 90.0 |
| 30 | kelt-16-b-S001-R001-C084-r.fit | LIGHT | 90.0 |
| 31 | kelt-16-b-S001-R001-C125-r.fit | LIGHT | 90.0 |
3.4.2.1. Decide what steps to take next#
In this case the only dark frames have exposure time 90 sec. Though that matches the exposure time of the science images, the flat field images are much shorter exposure time, ranging from 1 sec to 1.21 sec. This type of range of exposure is typical when twilight flats are taken. Since these are a much different exposure time than the darks, the dark frames will need to be scaled.
Recall that for this camera the overscan is not useful and should be trimmed off.
Given this, we will:
Trim the overscan from each of the dark frames.
Subtract calibration bias from the dark frames so that we can scale the darks to a different exposure time.
3.4.2.2. Calibration the individual dark frames#
First, we read the combined bias image created in the previous notebook. Though we could do this based on the file name, using a systematic set of header keywords to keep track of which images have been combined is less likely to lead to errors.
combined_bias = CCDData.read(ex2_images_reduced.files_filtered(imagetyp='bias',
combined=True,
include_path=True)[0])
for ccd, file_name in ex2_images_raw.ccds(imagetyp='DARK', # Just get the bias frames
return_fname=True # Provide the file name too.
):
# Trim the overscan
ccd = ccdp.trim_image(ccd[:, :4096])
# Subtract bias
ccd = ccdp.subtract_bias(ccd, combined_bias)
# Save the result
ccd.write(ex2_path_reduced / file_name)