Search

Calibrate dark images

Dark images, like any other images, need to be calibrated. Depending on the data you have and the choices you have made in reducing your data the steps to reducing your images may include:

  1. Subtracting overscan (only if you decide to subtract overscan from all images)
  2. Trim the image (if it has overscan, whether you are using the overscan or not)
  3. Subtract bias (if you need to scale the calibrated dark frames to a different exposure time).
from pathlib import Path

from astropy.nddata import CCDData
from ccdproc import ImageFileCollection
import ccdproc as ccdp

Example 1: Overscan subtracted, bias not removed

Take a look at what images you have

Click here to comment on this section on GitHub (opens in new tab).

First we gather up some information about the raw images and the reduced images up to this point. These examples have darks stored in a subdirectory of the folder with the rest of the images, so we create an ImageFileCollection for each.

ex1_path_raw = Path('example-cryo-LFC')

ex1_images_raw = ImageFileCollection(ex1_path_raw)
ex1_darks_raw = ImageFileCollection(ex1_path_raw / 'darks')

ex1_path_reduced = Path('example1-reduced')
ex1_images_reduced = ImageFileCollection(ex1_path_reduced)

Raw images, everything except the darks

ex1_images_raw.summary['file', 'imagetyp', 'exptime', 'filter']
Table masked=True length=14
fileimagetypexptimefilter
str14str9float64str2
ccd.001.0.fitsBIAS0.0i'
ccd.002.0.fitsBIAS0.0i'
ccd.003.0.fitsBIAS0.0i'
ccd.004.0.fitsBIAS0.0i'
ccd.005.0.fitsBIAS0.0i'
ccd.006.0.fitsBIAS0.0i'
ccd.014.0.fitsFLATFIELD70.001g'
ccd.015.0.fitsFLATFIELD70.011g'
ccd.016.0.fitsFLATFIELD70.001g'
ccd.017.0.fitsFLATFIELD7.0i'
ccd.018.0.fitsFLATFIELD7.0i'
ccd.019.0.fitsFLATFIELD7.0i'
ccd.037.0.fitsOBJECT300.062g'
ccd.043.0.fitsOBJECT300.014i'

Raw dark frames

ex1_darks_raw.summary['file', 'imagetyp', 'exptime', 'filter']
Table masked=True length=10
fileimagetypexptimefilter
str14str4float64str2
ccd.002.0.fitsBIAS0.0r'
ccd.013.0.fitsDARK300.0r'
ccd.014.0.fitsDARK300.0r'
ccd.015.0.fitsDARK300.0r'
ccd.017.0.fitsDARK70.0r'
ccd.018.0.fitsDARK70.0r'
ccd.019.0.fitsDARK70.0r'
ccd.023.0.fitsDARK7.0r'
ccd.024.0.fitsDARK7.0r'
ccd.025.0.fitsDARK7.0r'

Decide which calibration steps to take

Click here to comment on this section on GitHub (opens in new tab).

This example is, again, one of the chips of the LFC camera at Palomar. In earlier notebooks we have seen that the chip has a useful overscan region (LINK), has little dark current except for some hot pixels and sensor glow in one corner of the chip.

Looking at the list of non-dark images (i.e. the flat and light images) shows that for each exposure time in the non-dark images there is a set of dark exposures that has a matching, or very close to matching, exposure time.

To be more explicit, there are flats with exposure times of 7.0 sec and 70.011 sec and darks with exposure time of 7.0 and 70.0 sec. The dark and flat exposure times are close enough that there is no need to scale them. The two images of an object are each roughly 300 sec, matching the darks with exposure time 300 sec. The very small difference in exposure time, under 0.1 sec, does not need to be compensated for.

Given this, we will:

  1. Subtract overscan from each of the darks. The useful overscan region is XXX (see LINK).
  2. Trim the overscan out of the dark images

We will not subtract bias from these images because we will not need to rescale them to a different exposure time.

Calibrate the individual dark frames

for ccd, file_name in ex1_darks_raw.ccds(imagetyp='DARK',            # Just get the dark frames
                                         ccd_kwargs={'unit': 'adu'}, # CCDData requires a unit for the image if 
                                                                     # it is not in the header
                                         return_fname=True           # Provide the file name too.
                                        ):    
    # Subtract the overscan
    ccd = ccdp.subtract_overscan(ccd, overscan=ccd[:, 2055:], median=True)
    
    # Trim the overscan
    ccd = ccdp.trim_image(ccd[:, :2048])
    
    # Save the result
    ccd.write(ex1_path_reduced / file_name)

Reduced images (so far)

ex1_images_reduced.refresh()
ex1_images_reduced.summary['file', 'imagetyp', 'exptime', 'filter', 'combined']
Table masked=True length=16
fileimagetypexptimefiltercombined
str17str4float64str2object
ccd.001.0.fitsBIAS0.0i'--
ccd.002.0.fitsBIAS0.0i'--
ccd.003.0.fitsBIAS0.0i'--
ccd.004.0.fitsBIAS0.0i'--
ccd.005.0.fitsBIAS0.0i'--
ccd.006.0.fitsBIAS0.0i'--
ccd.013.0.fitsDARK300.0r'--
ccd.014.0.fitsDARK300.0r'--
ccd.015.0.fitsDARK300.0r'--
ccd.017.0.fitsDARK70.0r'--
ccd.018.0.fitsDARK70.0r'--
ccd.019.0.fitsDARK70.0r'--
ccd.023.0.fitsDARK7.0r'--
ccd.024.0.fitsDARK7.0r'--
ccd.025.0.fitsDARK7.0r'--
combined_bias.fitBIAS0.0i'True

Example 2: Overscan not subtracted, bias is removed

ex2_path_raw = Path('example-thermo-electric')

ex2_images_raw = ImageFileCollection(ex2_path_raw)

ex2_path_reduced = Path('example2-reduced')
ex2_images_reduced = ImageFileCollection(ex2_path_reduced)

We begin by looking at what exposure times we have in this data.

ex2_images_raw.summary['file', 'imagetyp', 'exposure'].show_in_notebook()
Table masked=True length=33
idxfileimagetypexposure
0AutoFlat-PANoRot-r-Bin1-001.fitFLAT1.0
1AutoFlat-PANoRot-r-Bin1-002.fitFLAT1.0
2AutoFlat-PANoRot-r-Bin1-003.fitFLAT1.0
3AutoFlat-PANoRot-r-Bin1-004.fitFLAT1.0
4AutoFlat-PANoRot-r-Bin1-005.fitFLAT1.0
5AutoFlat-PANoRot-r-Bin1-006.fitFLAT1.02
6AutoFlat-PANoRot-r-Bin1-007.fitFLAT1.06
7AutoFlat-PANoRot-r-Bin1-008.fitFLAT1.11
8AutoFlat-PANoRot-r-Bin1-009.fitFLAT1.16
9AutoFlat-PANoRot-r-Bin1-010.fitFLAT1.21
10Bias-S001-R001-C001-NoFilt.fitBIAS0.0
11Bias-S001-R001-C002-NoFilt.fitBIAS0.0
12Bias-S001-R001-C003-NoFilt.fitBIAS0.0
13Bias-S001-R001-C004-NoFilt.fitBIAS0.0
14Bias-S001-R001-C005-NoFilt.fitBIAS0.0
15Bias-S001-R001-C006-NoFilt.fitBIAS0.0
16Bias-S001-R001-C007-NoFilt.fitBIAS0.0
17Bias-S001-R001-C008-NoFilt.fitBIAS0.0
18Bias-S001-R001-C009-NoFilt.fitBIAS0.0
19Bias-S001-R001-C020-NoFilt.fitBIAS0.0
20Dark-S001-R001-C001-NoFilt.fitDARK90.0
21Dark-S001-R001-C002-NoFilt.fitDARK90.0
22Dark-S001-R001-C003-NoFilt.fitDARK90.0
23Dark-S001-R001-C004-NoFilt.fitDARK90.0
24Dark-S001-R001-C005-NoFilt.fitDARK90.0
25Dark-S001-R001-C006-NoFilt.fitDARK90.0
26Dark-S001-R001-C007-NoFilt.fitDARK90.0
27Dark-S001-R001-C008-NoFilt.fitDARK90.0
28Dark-S001-R001-C009-NoFilt copy.fitDARK90.0
29Dark-S001-R001-C009-NoFilt.fitDARK90.0
30Dark-S001-R001-C020-NoFilt.fitDARK90.0
31kelt-16-b-S001-R001-C084-r.fitLIGHT90.0
32kelt-16-b-S001-R001-C125-r.fitLIGHT90.0

Decide what steps to take next

Click here to comment on this section on GitHub (opens in new tab).

In this case the only dark frames have exposure time 90 sec. Though that matches the exposure time of the science images, the flat field images are much shorter exposure time, ranging from 1 sec to 1.21 sec. That type range of exposure is typical when twilights flats are taken. Since these are a much different exposure time than the darks, the dark frames will need to be scaled.

Recall that for this camera the overscan is not useful and should simply be trimmed off.

Given this, we will:

  1. Trim the overscan from each of the dark frames.
  2. Subtract calibration bias from the dark frames so that we can scale the darks to a different exposure time.

Calibration the individual dark frames

First, we read the combined bias image created in the previous notebook. Though we could do this based on the file name, using a systematic set of header keywords to keep track of which images have been combined is less likely to lead to errors.

combined_bias = CCDData.read(ex2_images_reduced.files_filtered(imagetyp='bias', 
                                                               combined=True, 
                                                               include_path=True)[0])
for ccd, file_name in ex2_images_raw.ccds(imagetyp='DARK',            # Just get the bias frames
                                          return_fname=True           # Provide the file name too.
                                         ):
        
    # Trim the overscan
    ccd = ccdp.trim_image(ccd[:, :4096])
    
    # Subtract bias
    ccd = ccdp.subtract_bias(ccd, combined_bias)
    # Save the result
    ccd.write(ex2_path_reduced / file_name)