-
Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models
Authors:
Annie S. Chen,
Alec M. Lessing,
Andy Tang,
Govind Chada,
Laura Smith,
Sergey Levine,
Chelsea Finn
Abstract:
Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. For example, in a search and rescue mission, a legged robot could climb over debris, crawl through gaps, and navigate out of dead ends. However, the robot's controller needs to respond intelligently to such varied obstacles, and this requires handling unexpected and unu…
▽ More
Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. For example, in a search and rescue mission, a legged robot could climb over debris, crawl through gaps, and navigate out of dead ends. However, the robot's controller needs to respond intelligently to such varied obstacles, and this requires handling unexpected and unusual scenarios successfully. This presents an open challenge to current learning methods, which often struggle with generalization to the long tail of unexpected situations without heavy human supervision. To address this issue, we investigate how to leverage the broad knowledge about the structure of the world and commonsense reasoning capabilities of vision-language models (VLMs) to aid legged robots in handling difficult, ambiguous situations. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection with VLMs: (1) in-context adaptation over previous robot interactions and (2) planning multiple skills into the future and replanning. We evaluate VLM-PC on several challenging real-world obstacle courses, involving dead ends and climbing and crawling, on a Go1 quadruped robot. Our experiments show that by reasoning over the history of interactions and future plans, VLMs enable the robot to autonomously perceive, navigate, and act in a wide range of complex scenarios that would otherwise require environment-specific engineering or human guidance.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
A 9-Month Hubble Space Telescope Near-UV Survey of M87. II. A Strongly Enhanced Nova Rate near the Jet of M87
Authors:
Alec M. Lessing,
Michael M. Shara,
Rebekah Hounsell,
Shifra Mandel,
Nava Feder,
William Sparks
Abstract:
The 135 classical novae that we have discovered in M87 with two $\textit{Hubble Space Telescope}$ imaging surveys appear to be strongly concentrated along that galaxy's jet. Detailed simulations show that the likelihood that this distribution occurred by chance is of order $0.3\%$. The novae near the jet display outburst characteristics (peak luminosities, colors and decline rates) that are indist…
▽ More
The 135 classical novae that we have discovered in M87 with two $\textit{Hubble Space Telescope}$ imaging surveys appear to be strongly concentrated along that galaxy's jet. Detailed simulations show that the likelihood that this distribution occurred by chance is of order $0.3\%$. The novae near the jet display outburst characteristics (peak luminosities, colors and decline rates) that are indistinguishable from novae far from the jet. We explore whether the remarkable nova distribution could be caused by the jet's irradiation of the hydrogen-rich donors in M87's cataclysmic binaries. This explanation, and others extant in the literature which rely on increased binary mass transfer rates, fail by orders of magnitude in explaining the enhanced nova rate near the jet. An alternate explanation is the presence of a genuine surplus of nova binary systems near the jet, perhaps due to jet-induced star formation. This explanation fails to explain the lack of nova enhancement along M87's counterjet. The enhanced rate of novae along M87's jet is now firmly established, and unexplained.
△ Less
Submitted 14 August, 2024; v1 submitted 28 September, 2023;
originally announced September 2023.
-
A 9-Month Hubble Space Telescope Near-UV Survey of M87. I. Light and Color Curves of 94 Novae, and a Re-determination of the Nova Rate
Authors:
Michael M. Shara,
Alec M. Lessing,
Rebekah Hounsell,
Shifra Mandel,
David Zurek,
Matthew J. Darnley,
Or Graur,
Yael Hillman,
Eileen T. Meyer,
Joanna Mikolajewska,
James D. Neill,
Dina Prialnik,
William Sparks
Abstract:
M87 has been monitored with a cadence of 5 days over a 9 month-long span through the near-ultraviolet (NUV:F275W) and optical (F606W) filters of the Wide Field Camera 3 (WFC3) of the $\textit{Hubble Space Telescope}$. This unprecedented dataset yields the NUV and optical light and color curves of 94 M87 novae, characterizing the outburst and decline properties of the largest extragalactic nova dat…
▽ More
M87 has been monitored with a cadence of 5 days over a 9 month-long span through the near-ultraviolet (NUV:F275W) and optical (F606W) filters of the Wide Field Camera 3 (WFC3) of the $\textit{Hubble Space Telescope}$. This unprecedented dataset yields the NUV and optical light and color curves of 94 M87 novae, characterizing the outburst and decline properties of the largest extragalactic nova dataset in the literature (after M31 and M81). We test and confirm nova modelers' prediction that recurrent novae cannot erupt more frequently that once every 45 days; show that there are zero rapidly recurring novae in the central $\sim$ 1/3 of M87 with recurrence times $ < $ 130 days; demonstrate that novae closely follow the K-band light of M87 to within a few arcsec of the galaxy nucleus; show that nova NUV light curves are as heterogeneous as their optical counterparts, and usually peak 5 to 30 days after visible light maximum; determine our observations' annual detection completeness to be 71 - 77\%; and measure the rate Rnova of nova eruptions in M87 as $352_{-37}^{+37}$/yr. The corresponding luminosity-specific classical nova rate for this galaxy is $7.91_{-1.20}^{+1.20}/yr/10^{10}L_\odot,_{K}$. These rates confirm that ground-based observations of extragalactic novae miss most faint, fast novae and those near the centers of galaxies. An annual M87 nova rate of 300 or more seems inescapable. A luminosity-specific nova rate of $\sim$ $7 - 10/yr/10^{10}L_\odot,_{K}$ in ${\it all}$ types of galaxies is indicated by the data available in 2023.
△ Less
Submitted 9 October, 2023; v1 submitted 29 August, 2023;
originally announced August 2023.
-
PHANGS-JWST First Results: The Dust Filament Network of NGC 628 and its Relation to Star Formation Activity
Authors:
David A. Thilker,
Janice C. Lee,
Sinan Deger,
Ashley T. Barnes,
Frank Bigiel,
Médéric Boquien,
Yixian Cao,
Mélanie Chevance,
Daniel A. Dale,
Oleg V. Egorov,
Simon C. O. Glover,
Kathryn Grasha,
Jonathan D. Henshaw,
Ralf S. Klessen,
Eric Koch,
J. M. Diederik Kruijssen,
Adam K. Leroy,
Ryan A. Lessing,
Sharon E. Meidt,
Francesca Pinna,
Miguel Querejeta,
Erik Rosolowsky,
Karin M. Sandstrom,
Eva Schinnerer,
Rowan J. Smith
, et al. (14 additional authors not shown)
Abstract:
PHANGS-JWST mid-infrared (MIR) imaging of nearby spiral galaxies has revealed ubiquitous filaments of dust emission in intricate detail. We present a pilot study to systematically map the dust filament network (DFN) at multiple scales between 25-400 pc in NGC 628. MIRI images at 7.7, 10, 11.3 and 21$μ$m of NGC 628 are used to generate maps of the filaments in emission, while PHANGS-HST B-band imag…
▽ More
PHANGS-JWST mid-infrared (MIR) imaging of nearby spiral galaxies has revealed ubiquitous filaments of dust emission in intricate detail. We present a pilot study to systematically map the dust filament network (DFN) at multiple scales between 25-400 pc in NGC 628. MIRI images at 7.7, 10, 11.3 and 21$μ$m of NGC 628 are used to generate maps of the filaments in emission, while PHANGS-HST B-band imaging yields maps of dust attenuation features. We quantify the correspondence between filaments traced by MIR thermal continuum / polycyclic aromatic hydrocarbon (PAH) emission and filaments detected via extinction / scattering of visible light; the fraction of MIR flux contained in the DFN; and the fraction of HII regions, young star clusters and associations within the DFN. We examine the dependence of these quantities with the physical scale at which the DFN is extracted. With our highest resolution DFN maps (25 pc filament width), we find that filaments in emission and attenuation are co-spatial in 40% of sight lines, often exhibiting detailed morphological agreement; that ~30% of the MIR flux is associated with the DFN; and that 75-80% of HII regions and 60% of star clusters younger than 5 Myr are contained within the DFN. However, the DFN at this scale is anti-correlated with looser associations of stars younger than 5 Myr identified using PHANGS-HST near-UV imaging. We discuss the impact of these findings for studies of star formation and the ISM, and the broad range of new investigations enabled with multi-scale maps of the DFN.
△ Less
Submitted 2 January, 2023;
originally announced January 2023.
-
A Hubble Space Telescope Survey for Novae in the Globular Clusters of M87
Authors:
Tricia F. Doyle,
Michael M. Shara,
Alec Lessing,
David Zurek
Abstract:
The giant elliptical galaxy M87 has been imaged over 30 consecutive days in 2001, 60 consecutive days in 2005-2006, and every 5 days over a 265 day span in 2016-2017 with the Hubble Space Telescope, leading to the detection of 137 classical novae throughout M87. We have identified 2134 globular clusters (GC) in M87 in these images, and carried out searches of the clusters for classical novae erupt…
▽ More
The giant elliptical galaxy M87 has been imaged over 30 consecutive days in 2001, 60 consecutive days in 2005-2006, and every 5 days over a 265 day span in 2016-2017 with the Hubble Space Telescope, leading to the detection of 137 classical novae throughout M87. We have identified 2134 globular clusters (GC) in M87 in these images, and carried out searches of the clusters for classical novae erupting in or near them. One GC CN was detected in the 2001 data, while zero novae were found during the 2005-2006 observations. Four candidate GC novae were (barely) detected in visible light during the 2016-2017 observations, but none of the four were seen in near-ultraviolet light, leading us to reject them. Combining these results with our detection of one M87 GC nova out of a total of 137 detected CN, we conclude that such novae may be overabundant relative to the field, but small number statistics dominate this (and all other) searches. A definitive determination of GC CN overabundance (or not) will require much larger samples which LSST should provide in the coming decade.
△ Less
Submitted 29 October, 2018;
originally announced October 2018.
-
A 2D IR Study of Isotope-Edited Variants of the Elastin-like GVGVPGVG Peptide and the Size Dependent Behavior of (VPGVG)n
Authors:
Joshua A. Lessing
Abstract:
This study uses two-dimensional infrared (2D IR) spectroscopy in conjunction with isotope labeling and spectral modeling from molecular dynamics simulations to identify the dominant turn conformations that exist in equilibrium ensembles of the (VPGVG)n family of intrinsically disordered elastin-like peptides. Numerous models have been proposed to explain the origins of elastin's elasticity and its…
▽ More
This study uses two-dimensional infrared (2D IR) spectroscopy in conjunction with isotope labeling and spectral modeling from molecular dynamics simulations to identify the dominant turn conformations that exist in equilibrium ensembles of the (VPGVG)n family of intrinsically disordered elastin-like peptides. Numerous models have been proposed to explain the origins of elastin's elasticity and its counterintuitive ability to become structured upon heating. However, the structure of elastin remains unassigned because none of the techniques currently used to study highly disordered amino acid sequences have a time resolution that is faster than the lifetime of a transient conformation in a disordered sequence. Because these conformations exchange on time scales longer than the 5-6 ps required for 2D IR measurements, isotope-edited 2D IR spectroscopy was chosen to study this family of peptides. This study first examined the small GVGVPGVG peptide at a series of temperatures and salt concentrations to assign its dominant turn conformations and to determine what variables alter the populations of these conformations. These data were then used to identify the dominant turn conformations that exist in larger elastin-like peptides including (GVGVP)251. The results indicate that the (VPGVG)n family of elastin-like peptides contain a high population of both an irregular turn structure with 2 peptide-peptide hydrogen bonds to the proline amide C=O group and a conventional turn structure with 1 peptide-peptide hydrogen bond to the proline amide C=O group. These turn structures are durable, showing a significant population of closed turns at all temperatures and salt concentrations studied.
△ Less
Submitted 19 September, 2017; v1 submitted 11 September, 2017;
originally announced September 2017.
