Skip to content

Build 18v1 #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 4, 2025
Merged

Build 18v1 #39

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b3f0243
Initial commit
PaulHuwe Jun 3, 2025
610231b
Corrected STPSF location
PaulHuwe Jun 3, 2025
3aec343
Adjusted CRDS pointer.
PaulHuwe Jun 3, 2025
fcbeb66
Merge branch 'main' into Build_18v1
PaulHuwe Jun 3, 2025
00b3c4e
Merge branch 'main' of https://github.com/spacetelescope/roman-data-w…
PaulHuwe Jun 3, 2025
6b3eb4b
Removed mentioned of Drizzlepac
PaulHuwe Jun 3, 2025
99284b6
Merge branch 'Build_18v1' of https://github.com/PaulHuwe/roman-data-w…
PaulHuwe Jun 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 59 additions & 70 deletions 10-Mosaic_Pipeline/10-Mosaic_Pipeline.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -111,7 +111,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 3,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -143,7 +143,7 @@
"}"
]
},
"execution_count": 4,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -161,7 +161,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -179,7 +179,7 @@
"\n",
"Next we run the Mosaic Pipeline using the `MosaicPipeline` class. As with the L1 to L2 Exposure Pipeline, there are many optional arguments to customize the behavior of the pipeline. At this time, we will not delve into these optional parameters. The simulated input files are based on a simple gap-filling dither pattern rather than sub-pixel dithering optimized for the point spread function(PSF). As a result, we use the default native sampling of the WFI detectors. In the future, additional details may be provided to explore optimizing the Mosaic Pipeline output.\n",
"\n",
"If we had only a single input image, this is where we would skip the skymatch and outlier detection steps by passing the optional argument `steps={'skymatch': {'skip': True}, 'outlier_detection': {'skip': True}}`. For now, we will skip the source catalog step that would normally make the single-band source catalog and segmentation map, which are Level 4 (L4; high-level extracted information) files.\n",
"As we have only a single input image, this is where we skip the skymatch and outlier detection steps by passing the optional argument `steps={'skymatch': {'skip': True}, 'outlier_detection': {'skip': True}}`. Inaddition, we will skip the source catalog step that would normally make the single-band source catalog and segmentation map, which are Level 4 (L4; high-level extracted information) files.\n",
"\n",
"**IMPORTANT NOTE:** At this time, L4 products are still being developed and validated, and we expect significant changes in their format and the algorithms used to generate them. We do not recommend the use of these products until they are fully validated.\n",
"\n",
Expand All @@ -188,27 +188,29 @@
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"execution_count": 5,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025-06-02 23:55:33,205 - CRDS - ERROR - Error determining best reference for 'pars-fluxstep' = No match found.\n",
"2025-06-02 23:55:33,208 - CRDS - ERROR - Error determining best reference for 'pars-skymatchstep' = No match found.\n",
"2025-06-02 23:55:33,211 - CRDS - ERROR - Error determining best reference for 'pars-outlierdetectionstep' = No match found.\n",
"2025-06-02 23:55:33,214 - CRDS - ERROR - Error determining best reference for 'pars-resamplestep' = No match found.\n",
"2025-06-02 23:55:33,216 - CRDS - ERROR - Error determining best reference for 'pars-sourcecatalogstep' = Unknown reference type 'pars-sourcecatalogstep'\n",
"2025-06-02 23:55:33,218 - CRDS - ERROR - Error determining best reference for 'pars-mosaicpipeline' = Unknown reference type 'pars-mosaicpipeline'\n",
"2025-06-02 23:55:33,226 - stpipe.MosaicPipeline - INFO - MosaicPipeline instance created.\n",
"2025-06-02 23:55:33,227 - stpipe.MosaicPipeline.flux - INFO - FluxStep instance created.\n",
"2025-06-02 23:55:33,228 - stpipe.MosaicPipeline.skymatch - INFO - SkyMatchStep instance created.\n",
"2025-06-02 23:55:33,230 - stpipe.MosaicPipeline.outlier_detection - INFO - OutlierDetectionStep instance created.\n",
"2025-06-02 23:55:33,232 - stpipe.MosaicPipeline.resample - INFO - ResampleStep instance created.\n",
"2025-06-02 23:55:33,233 - stpipe.MosaicPipeline.sourcecatalog - INFO - SourceCatalogStep instance created.\n",
"2025-06-02 23:55:33,293 - stpipe.MosaicPipeline - INFO - Step MosaicPipeline running with args ('my_roman_mosaic_asn.json',).\n",
"2025-06-02 23:55:33,297 - stpipe.MosaicPipeline - INFO - Step MosaicPipeline parameters are:\n",
"2025-06-03 18:34:00,394 - CRDS - ERROR - Error determining best reference for 'pars-fluxstep' = No match found.\n",
"2025-06-03 18:34:00,397 - CRDS - ERROR - Error determining best reference for 'pars-skymatchstep' = No match found.\n",
"2025-06-03 18:34:00,400 - CRDS - ERROR - Error determining best reference for 'pars-outlierdetectionstep' = No match found.\n",
"2025-06-03 18:34:00,403 - CRDS - ERROR - Error determining best reference for 'pars-resamplestep' = No match found.\n",
"2025-06-03 18:34:00,406 - CRDS - ERROR - Error determining best reference for 'pars-sourcecatalogstep' = Unknown reference type 'pars-sourcecatalogstep'\n",
"2025-06-03 18:34:00,408 - CRDS - ERROR - Error determining best reference for 'pars-mosaicpipeline' = Unknown reference type 'pars-mosaicpipeline'\n",
"2025-06-03 18:34:00,417 - stpipe.MosaicPipeline - INFO - MosaicPipeline instance created.\n",
"2025-06-03 18:34:00,418 - stpipe.MosaicPipeline.flux - INFO - FluxStep instance created.\n",
"2025-06-03 18:34:00,419 - stpipe.MosaicPipeline.skymatch - INFO - SkyMatchStep instance created.\n",
"2025-06-03 18:34:00,421 - stpipe.MosaicPipeline.outlier_detection - INFO - OutlierDetectionStep instance created.\n",
"2025-06-03 18:34:00,422 - stpipe.MosaicPipeline.resample - INFO - ResampleStep instance created.\n",
"2025-06-03 18:34:00,424 - stpipe.MosaicPipeline.sourcecatalog - INFO - SourceCatalogStep instance created.\n",
"2025-06-03 18:34:00,484 - stpipe.MosaicPipeline - INFO - Step MosaicPipeline running with args ('my_roman_mosaic_asn.json',).\n",
"2025-06-03 18:34:00,489 - stpipe.MosaicPipeline - INFO - Step MosaicPipeline parameters are:\n",
" pre_hooks: []\n",
" post_hooks: []\n",
" output_file: None\n",
Expand Down Expand Up @@ -246,7 +248,7 @@
" output_use_model: False\n",
" output_use_index: True\n",
" save_results: False\n",
" skip: False\n",
" skip: True\n",
" suffix: None\n",
" search_output_file: True\n",
" input_dir: ''\n",
Expand All @@ -271,7 +273,7 @@
" output_use_model: False\n",
" output_use_index: True\n",
" save_results: False\n",
" skip: False\n",
" skip: True\n",
" suffix: None\n",
" search_output_file: True\n",
" input_dir: ''\n",
Expand Down Expand Up @@ -335,51 +337,38 @@
" deblend: False\n",
" fit_psf: True\n",
" forced_segmentation: ''\n",
"2025-06-02 23:55:33,298 - stpipe.MosaicPipeline - INFO - Starting Roman mosaic level calibration pipeline ...\n",
"2025-06-02 23:55:33,354 - stpipe.MosaicPipeline.flux - INFO - Step flux running with args (<romancal.datamodels.library.ModelLibrary object at 0x15708f310>,).\n",
"2025-06-02 23:55:33,781 - stpipe.MosaicPipeline.flux - INFO - Step flux done\n",
"2025-06-02 23:55:33,832 - stpipe.MosaicPipeline.skymatch - INFO - Step skymatch running with args (<romancal.datamodels.library.ModelLibrary object at 0x15708f310>,).\n",
"2025-06-02 23:55:33,891 - stpipe.MosaicPipeline.skymatch - INFO - \n",
"2025-06-02 23:55:33,891 - stpipe.MosaicPipeline.skymatch - INFO - ***** stcal.skymatch.skymatch.skymatch() started on 2025-06-02 23:55:33.891331\n",
"2025-06-02 23:55:33,891 - stpipe.MosaicPipeline.skymatch - INFO - \n",
"2025-06-02 23:55:33,892 - stpipe.MosaicPipeline.skymatch - INFO - Sky computation method: 'match'\n",
"2025-06-02 23:55:33,892 - stpipe.MosaicPipeline.skymatch - INFO - Sky matching direction: DOWN\n",
"2025-06-02 23:55:33,892 - stpipe.MosaicPipeline.skymatch - INFO - Sky subtraction from image data: OFF\n",
"2025-06-02 23:55:33,892 - stpipe.MosaicPipeline.skymatch - INFO - \n",
"2025-06-02 23:55:33,892 - stpipe.MosaicPipeline.skymatch - INFO - ---- Computing differences in sky values in overlapping regions.\n",
"2025-06-02 23:55:33,893 - stpipe.MosaicPipeline.skymatch - WARNING - * Image ID=r0000101001001001001_0001_wfi01_f158_cal.asdf: Unable to compute sky value\n",
"2025-06-02 23:55:33,893 - stpipe.MosaicPipeline.skymatch - INFO - * Image ID=r0000101001001001001_0001_wfi01_f158_cal.asdf. Sky background: 0\n",
"2025-06-02 23:55:33,893 - stpipe.MosaicPipeline.skymatch - INFO - \n",
"2025-06-02 23:55:33,893 - stpipe.MosaicPipeline.skymatch - INFO - ***** stcal.skymatch.skymatch.skymatch() ended on 2025-06-02 23:55:33.893545\n",
"2025-06-02 23:55:33,893 - stpipe.MosaicPipeline.skymatch - INFO - ***** stcal.skymatch.skymatch.skymatch() TOTAL RUN TIME: 0:00:00.002214\n",
"2025-06-02 23:55:33,894 - stpipe.MosaicPipeline.skymatch - INFO - \n",
"2025-06-02 23:55:33,895 - stpipe.MosaicPipeline.skymatch - INFO - Step skymatch done\n",
"2025-06-02 23:55:33,947 - stpipe.MosaicPipeline.outlier_detection - INFO - Step outlier_detection running with args (<romancal.datamodels.library.ModelLibrary object at 0x15708f310>,).\n",
"2025-06-02 23:55:33,947 - stpipe.MosaicPipeline.outlier_detection - WARNING - Skipping outlier_detection - at least two imaging observations are needed.\n",
"2025-06-02 23:55:33,949 - stpipe.MosaicPipeline.outlier_detection - INFO - Step outlier_detection done\n",
"2025-06-02 23:55:33,996 - stpipe.MosaicPipeline.resample - INFO - Step resample running with args (<romancal.datamodels.library.ModelLibrary object at 0x15708f310>,).\n",
"2025-06-02 23:55:34,009 - stpipe.MosaicPipeline.resample - INFO - Output pixel scale: 0.10930713430447905 arcsec.\n",
"2025-06-02 23:55:34,009 - stpipe.MosaicPipeline.resample - INFO - Driz parameter kernel: square\n",
"2025-06-02 23:55:34,010 - stpipe.MosaicPipeline.resample - INFO - Driz parameter pixfrac: 1.0\n",
"2025-06-02 23:55:34,010 - stpipe.MosaicPipeline.resample - INFO - Driz parameter fillval: NAN\n",
"2025-06-02 23:55:34,010 - stpipe.MosaicPipeline.resample - INFO - Driz parameter weight_type: ivm\n",
"2025-06-02 23:55:38,378 - stpipe.MosaicPipeline.resample - INFO - Resampling science and variance data\n",
"2025-06-02 23:55:43,193 - stpipe.MosaicPipeline.resample - INFO - Drizzling (4088, 4088) --> (4060, 4138)\n",
"2025-06-02 23:55:45,864 - stpipe.MosaicPipeline.resample - INFO - Drizzling (4088, 4088) --> (4060, 4138)\n",
"2025-06-02 23:55:48,541 - stpipe.MosaicPipeline.resample - INFO - Drizzling (4088, 4088) --> (4060, 4138)\n",
"2025-06-02 23:55:51,420 - stpipe.MosaicPipeline.resample - INFO - Mean, max exposure times: 130.7, 130.7\n",
"2025-06-02 23:55:51,436 - stpipe.MosaicPipeline.resample - INFO - Update S_REGION to POLYGON ICRS 269.986983065 65.974249689 269.986919794 66.097523722 269.676833272 66.097186783 269.678393892 65.973914377\n",
"2025-06-02 23:55:51,446 - stpipe.MosaicPipeline.resample - INFO - Step resample done\n",
"2025-06-02 23:55:51,498 - stpipe.MosaicPipeline.sourcecatalog - INFO - Step sourcecatalog running with args (<roman_datamodels.datamodels._datamodels.MosaicModel object at 0x1572cfdd0>,).\n",
"2025-06-02 23:55:51,498 - stpipe.MosaicPipeline.sourcecatalog - INFO - Step skipped.\n",
"2025-06-02 23:55:51,500 - stpipe.MosaicPipeline.sourcecatalog - INFO - Could not record skip into DataModel header: No such attribute (source_catalog) found in node\n",
"2025-06-02 23:55:52,688 - stpipe.MosaicPipeline - INFO - Saved model in my_roman_mosaic_coadd.asdf\n",
"2025-06-02 23:55:52,689 - stpipe.MosaicPipeline - INFO - Step MosaicPipeline done\n"
"2025-06-03 18:34:00,489 - stpipe.MosaicPipeline - INFO - Starting Roman mosaic level calibration pipeline ...\n",
"2025-06-03 18:34:00,544 - stpipe.MosaicPipeline.flux - INFO - Step flux running with args (<romancal.datamodels.library.ModelLibrary object at 0x1663a6e50>,).\n",
"2025-06-03 18:34:00,973 - stpipe.MosaicPipeline.flux - INFO - Step flux done\n",
"2025-06-03 18:34:01,024 - stpipe.MosaicPipeline.skymatch - INFO - Step skymatch running with args (<romancal.datamodels.library.ModelLibrary object at 0x1663a6e50>,).\n",
"2025-06-03 18:34:01,024 - stpipe.MosaicPipeline.skymatch - INFO - Step skipped.\n",
"2025-06-03 18:34:01,073 - stpipe.MosaicPipeline.outlier_detection - INFO - Step outlier_detection running with args (<romancal.datamodels.library.ModelLibrary object at 0x1663a6e50>,).\n",
"2025-06-03 18:34:01,073 - stpipe.MosaicPipeline.outlier_detection - INFO - Step skipped.\n",
"2025-06-03 18:34:01,121 - stpipe.MosaicPipeline.resample - INFO - Step resample running with args (<romancal.datamodels.library.ModelLibrary object at 0x1663a6e50>,).\n",
"2025-06-03 18:34:01,144 - stpipe.MosaicPipeline.resample - INFO - Output pixel scale: 0.10930713430447905 arcsec.\n",
"2025-06-03 18:34:01,145 - stpipe.MosaicPipeline.resample - INFO - Driz parameter kernel: square\n",
"2025-06-03 18:34:01,145 - stpipe.MosaicPipeline.resample - INFO - Driz parameter pixfrac: 1.0\n",
"2025-06-03 18:34:01,145 - stpipe.MosaicPipeline.resample - INFO - Driz parameter fillval: NAN\n",
"2025-06-03 18:34:01,145 - stpipe.MosaicPipeline.resample - INFO - Driz parameter weight_type: ivm\n",
"2025-06-03 18:34:05,525 - stpipe.MosaicPipeline.resample - INFO - Resampling science and variance data\n",
"2025-06-03 18:34:10,415 - stpipe.MosaicPipeline.resample - INFO - Drizzling (4088, 4088) --> (4060, 4138)\n",
"2025-06-03 18:34:13,131 - stpipe.MosaicPipeline.resample - INFO - Drizzling (4088, 4088) --> (4060, 4138)\n",
"2025-06-03 18:34:15,785 - stpipe.MosaicPipeline.resample - INFO - Drizzling (4088, 4088) --> (4060, 4138)\n",
"2025-06-03 18:34:18,663 - stpipe.MosaicPipeline.resample - INFO - Mean, max exposure times: 130.7, 130.7\n",
"2025-06-03 18:34:18,679 - stpipe.MosaicPipeline.resample - INFO - Update S_REGION to POLYGON ICRS 269.986983065 65.974249689 269.986919794 66.097523722 269.676833272 66.097186783 269.678393892 65.973914377\n",
"2025-06-03 18:34:18,689 - stpipe.MosaicPipeline.resample - INFO - Step resample done\n",
"2025-06-03 18:34:18,742 - stpipe.MosaicPipeline.sourcecatalog - INFO - Step sourcecatalog running with args (<roman_datamodels.datamodels._datamodels.MosaicModel object at 0x16e63cf90>,).\n",
"2025-06-03 18:34:18,742 - stpipe.MosaicPipeline.sourcecatalog - INFO - Step skipped.\n",
"2025-06-03 18:34:18,744 - stpipe.MosaicPipeline.sourcecatalog - INFO - Could not record skip into DataModel header: No such attribute (source_catalog) found in node\n",
"2025-06-03 18:34:19,847 - stpipe.MosaicPipeline - INFO - Saved model in my_roman_mosaic_coadd.asdf\n",
"2025-06-03 18:34:19,848 - stpipe.MosaicPipeline - INFO - Step MosaicPipeline done\n"
]
}
],
"source": [
"result = MosaicPipeline.call('my_roman_mosaic_asn.json', save_results=True, steps={'sourcecatalog': {'skip': True}})"
"result = MosaicPipeline.call('my_roman_mosaic_asn.json', save_results=True, steps={'sourcecatalog': {'skip': True},\n",
" 'skymatch': {'skip': True},\n",
" 'outlier_detection': {'skip': True}})"
]
},
{
Expand All @@ -391,7 +380,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 6,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -433,16 +422,16 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.image.AxesImage at 0x157305310>"
"<matplotlib.image.AxesImage at 0x177b09b90>"
]
},
"execution_count": 8,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
},
Expand Down Expand Up @@ -473,7 +462,7 @@
"- err: The error image\n",
"- weight: A weight map showing the relative coverage on the sky from the input images and how they contributed to the final mosaic\n",
"\n",
"For more information on the how arrays are computed by the drizzle algorithm, we suggest consulting the [DrizzlePac Handbook](https://www.stsci.edu/files/live/sites/www/files/home/scientific-community/software/drizzlepac/_documents/drizzlepac-handbook.pdf). While the DrizzlePac Handbook is written for HST, many of the same terms and principles apply to Roman as the underlying algorithm is the same."
"For more information on the how arrays are computed by the drizzle algorithm, we suggest consulting the [Resample Doc](https://roman-pipeline.readthedocs.io/en/latest/roman/resample/main.html). "
]
},
{
Expand All @@ -483,7 +472,7 @@
"## Additional Resources\n",
"- [RomanCal Documentation](https://roman-pipeline.readthedocs.io/en/latest/index.html)\n",
"- [Roman Documentation](https://roman-docs.stsci.edu)\n",
"- [DrizzlePac Handbook](https://www.stsci.edu/files/live/sites/www/files/home/scientific-community/software/drizzlepac/_documents/drizzlepac-handbook.pdf)"
"- [Resample Doc](https://roman-pipeline.readthedocs.io/en/latest/roman/resample/main.html)"
]
},
{
Expand Down
Loading