Extending 3-nighters in Rubin's DP0.3 data to 4+ nights with Fourtify
Summary: Over 70% of solar system objects in Rubin's DP0.3 appear with 2 detections per night on only 3 nights in a 15 day HelioLinC search window. Since 3-nighters can sometimes produce spurious linkages, searching for more nights of data for an object beyond its 15 day HelioLinC search window is ideal. Fourtify uses the orbit of a 3-nighter findable by HelioLinC to search DP0.3's DiaSources for additional transient sources along that orbit beyond a 15 day HelioLinC search window. By searching the 45 days of data beyond a HelioLinC 15 day search window, Fourtify can extend 76.58% of all 3-nighters and 84.80% of extendable 3-nighters to 4 or more nights for a random sample of 7,087 objects in DP0.3 with only 3 nights of possible detections in their 15 day HelioLinC search window. The median number of nights an object is extended to is 7. The fraction of extensions to 4+ nights that are impure is 0.06%.
Motivation
For this analysis I split Rubin's DP0.3 DiaSource data into eight independent, non-overlapping 45 day windows spanning 360 days. I calculate that 72.80% of linkable objects1 in the central 15 day window of these eight 45 day windows are linkable with a maximum of 3 nights of data. That is, the longest linkage HelioLinC can possibly find in these 15 day windows is a 3-nighter 72.80% of the time. Since myself and others (Ari Heinze, Peter Veres and likely others that I am not aware of) have previously demonstrated that the orbits produced with only 3 nights of detections may occasionally be spurious, it’d be better to increase the nights of data for a new object to at least 4. This is not possible for 72.80% of DP0.3 objects unless you search beyond the 15 day HelioLinC search window.
Fourtify Search and Arc Extension Technique
Fourtify works by propagating the orbit calculated from the orbit determination of a linkage (the 3-nighter) to all unique observation times in the extended search window (45 days in this instance) then differencing those propagated positions on the sky from the transient sources at the same observation times to find transient sources within a specified angular tolerance of the orbit. I use Bill Gray’s Find_Orb for all orbit determination in this study.
If the number of transient sources returned by Fourtify spans at least 4 nights and there are at least two detections a minimum of 4 of those nights, the transient source set is OD solved. If the orbit solution is “good,” a 4+ nighter has been obtained from a 3-nighter. If the orbit solution is not good, we iteratively prune the Fourtify search’s transient set one source at a time (from highest angular deviation to lowest along the orbit searched) OD solving at each step until either a good orbit is produced or the set of transients falls below 3 nights of data. In the latter case, no extension to 4 nights occurs.
I will define a “good” linkage for this analysis as one with at least 4 nights of data and having at least 2 observations on each of those 4 nights and an RMS less than or equal to 0.15”. If the linkage has exactly 4 nights of observations, there is an additional filter applied such that no individual residual exceeds 0.15”. Singleton detections on a night are allowed only if there are at least 4 nights with 2+ detections per night.
DP0.3 Fourtify Search and Parameters
For this analysis I will compare two Fourtify 45 day search windows. The first is a trailing 45 day window where the HelioLinC 15 day search interval is at the end of that 45 day window and you are searching the transients that precede the HelioLinC search window in time. The second is a 45 day centered window where the HelioLinC 15 day search interval sits in the middle 15 days of a 45 day window. For both the trailing and centered Fourtify searches, the HelioLinC search window contains the 3-nighters that you use to generate an orbit to search along. The figure below illustrates these two Fourtify searches.
Figure 1: The Fourtify transient search window (in time) relative to the HelioLinC search window
In each of these windows I search for transient sources in DiaSources that have up to a specified maximum deviation from the search orbit that grows at a linear rate from a starting level of 10” at the orbit epoch + 2”/day up to a maximum deviation of 100”. I’m thus searching two cone-shaped regions that become a cylinder at their maximum deviation around the specified orbit with slightly lopped off vertices at the epoch of the OD solution.
Figure 2: The Fourtify search cone around the nominal orbit we're searching along to extend a 3-nighter to 4+ nights
There are hundreds of thousands of objects in DP0.3. Instead of testing all of these objects I took a random sample of up to 1,000 objects in each of the eight windows I defined above. In total I searched 7,087 objects with only 3 nights of data in their HelioLinC search window for extension to a 4th night in a 45 day window. Seven of the eight windows had many more than 1000 objects in them, but one had only 87. I double checked this value, but did not investigate deeply why there were only 87. Perhaps it was centered around a simulated full moon, or perhaps there was an error in modeling the cadence. I am only speculating here but I expect that the next release of data will likely not have this feature.
Finally, I should emphasize that when I say the HelioLinC 15 day search window I am not using HelioLinC to perform any kind of search for linkages anywhere in this study. I’m just taking the 3-nighters that HelioLinC could find and tryng to extend those 3-nighters with Fourtify beyond the 15 day HelioLinC search window.
Results
The results for the two searches (trailing and centered) described above are summarized in the table below. The takeaway is that the centered search performs better. The reason for this is fairly obvious. Objects in the trailing search are more likely to have been seen for the first time ever in their 15 day HelioLinC search window. Thus looking at past transients in the trailing search won’t reveal any more detections of the object.
| Trailing Extended | Trailing Extendable | Centered Extended | Centered Extendable | Extended in Either | Extendable in Either | 3-nighters Searched for Extension | |
|---|---|---|---|---|---|---|---|
| 3-nighters extended to 4+ nights |
3,584 | 4,321 | 5,226 | 6,240 | 5,427 | 6,400 | 7,087 |
Still, 50.57% of all 3-nighters in the trailing search can be extended to a 4th night with Fourtify. For the centered window that number rises to 73.74%. The union of both searches yields an extension of 76.58% objects to a 4th night in one or the other window. If we consider only objects that have at least 4 nights of data in the Fourtify search window, 84.80% of them are extended in either the trailing or centered window.
| Trailing Extended | Centered Extended | Either Trailing or Centered | |
|---|---|---|---|
| Fraction of Extendable Objects |
82.94% | 83.75% | 84.80% |
| Fraction of All Objects |
50.57% | 73.74% | 76.58% |
As a sort of sanity check, you can see in the table above that 3-nighters that can be extended – those that have a 4th night of data somewhere in their 45 day search window – are extended ~83% of the time for either the trailing or centered windows. You would expect this to be consistent and it is. The question is: why can’t we extend 100% of extendable objects? More on that below.
There’s one other very important metric to highlight. 2 out of the 3,584 objects extended to 4+ nights in the trailing window and 3 out of the 5,226 objects extended to 4+ nights in the centered window are impure extensions – that is they produced “good” orbit metrics but the sources belonged to more than one object (ssObjectId). Thus impure extensions represent 0.06% of the extensions that Fourtify finds. There are reasons to believe it will be relatively easy to zero out these impure extensions that I talk about in the discussion below.
One last metric that I think is very neat in these results: the median number of nights of data for a 3-nighter that was extended by Fourtify is 7 in the trailing window and 6.5 in the centered window.3 Turning 3-nighters into 6 or 7-nighters is a great improvement! That’s a lot of extra data to improve an orbit solution with.
Discussion and Next Steps
These results are from a study I did last summer and am only just now writing up. I have notes about two of the five impure extensions I mentioned above that are encouraging. At least one of the impure extensions contained an individual residual in excess of 0.15”. Filtering individual outliers is something that I do extensively for ITF identifications, and I should be doing it more here too (I’m already doing it for cases where the extension is exactly 4 nights). The other impure extension I have notes for was an 8-nighter but 4 of those nights had only singleton detections (one, not two or more detections on a night). I already require that at least 4 nights have 2 detections per night, but it might be better to only use nights with more than one detection in a final extended linkage if the individual residual filter alone doesn’t fix this.
There is one other explanation for these impure extensions. There are objects in DP0.3 that are unintentionally duplicated exactly but have different labels (different ssObjectIds). This is a known bug in DP0.3 data and it’s possible that my Fourtify searches found transients along the same exact orbit of the expected object where the label of one of the transients (the ssObjectId) was different than the starting 3-nighter label leading me to incorrectly classify the extension as impure. New data is forthcoming that will presumably not have this bug, so I can test this theory then.
Why can’t we extend 100% of extendable objects? Well, maybe we eventually can. I certainly think increasing the fraction is possible. But this problem also seems related to the intractable mislinkages issue to me.
I will preface this part as speculation that needs further investigation, but the 3-nighters that didn’t extend yet could have been extended (a 4th night did exist in the data and Fourtify just didn’t find it) often had two characteristics that I found noteworthy:
- They frequently had an RMS in excess of 0.15” on the 3-night linkage that produced the Fourtify search orbit. For the centered Fourtify search 65% of the extendable objects that were not extended had an RMS > 0.15”. Remember, all the objects in DP0.3 are labeled, so I’m calculating an orbit to search along from a 3-nighter on what I know to be a pure object. This means that the orbit for some 3-nighters doesn’t appear to represent itself as well as we would expect given Rubin’s astrometric accuracy of ~0.05”.
- 3-nighters that weren’t extended often had a small gap (in time) between two of their detection nights. For example, in the centered window Fourtify search ~68% of the objects that could be extended that were not extended had a small gap2 of less than 2.5 days. This small gap measure is one that has been observed to lead to intractables by myself and others. This is clearly not mutually exclusive with point 1.
As far as next steps go, I plan to re-run this study with Rubin’s forthcoming DP0.3.1 data (I don’t think it’s called that, but it’s the next iteration of the solar system Data Preview) using one or both of the tighter constraints on what constitutes a “good” orbit that I outlined in the section above. I do not think it will be difficult to zero out the impure extensions, but there may be some small trade-off in the number of objects that can be extended with tighter filters.
Acknowledgements
Thanks to Siegfried Eggl and Ari Heinze for feedback on this work and to Mario Juric for the unintentional nudge that led me to revisit my results and write them up here.
Credits
The DP0.3 data set was generated by members of the Rubin Solar System Pipelines and Commissioning teams, with help from the LSST Solar System Science Collaboration, in particular: Pedro Bernardinelli, Jake Kurlander, Joachim Moeyens, Samuel Cornwall, Ari Heinze, Steph Merritt, Lynne Jones, Siegfried Eggl, Meg Schwamb, Grigori Fedorets, and Mario Juric.
Footnotes
- An object is defined as linkable for this study if there are at least 2 detections per night over 3 nights and the object is moving at a rate of less than 10 deg/day and detections are separated (in time) on a night by no less than 5 minutes and no more than 90 minutes.
- The total arc, the difference between the time of the last observation and the first observation, is given by: arc = small gap + large gap for the case of a 3-nighter.
- Since there were 8 independent searches across the data these medians are actually the median of the 8 individual medians.
Published: 4/14/2025