-
Towards Human-Robot Teaming through Augmented Reality and Gaze-Based Attention Control
Authors:
Yousra Shleibik,
Elijah Alabi,
Christopher Reardon
Abstract:
Robots are now increasingly integrated into various real world applications and domains. In these new domains, robots are mostly employed to improve, in some ways, the work done by humans. So, the need for effective Human-Robot Teaming (HRT) capabilities grows. These capabilities usually involve the dynamic collaboration between humans and robots at different levels of involvement, leveraging the…
▽ More
Robots are now increasingly integrated into various real world applications and domains. In these new domains, robots are mostly employed to improve, in some ways, the work done by humans. So, the need for effective Human-Robot Teaming (HRT) capabilities grows. These capabilities usually involve the dynamic collaboration between humans and robots at different levels of involvement, leveraging the strengths of both to efficiently navigate complex situations. Crucial to this collaboration is the ability of robotic systems to adjust their level of autonomy to match the needs of the task and the human team members.
This paper introduces a system designed to control attention using HRT through the use of ground robots and augmented reality (AR) technology. Traditional methods of controlling attention, such as pointing, touch, and voice commands, sometimes fall short in precision and subtlety. Our system overcomes these limitations by employing AR headsets to display virtual visual markers. These markers act as dynamic cues to attract and shift human attention seamlessly, irrespective of the robot's physical location.
△ Less
Submitted 22 August, 2024;
originally announced August 2024.
-
Compositional Zero-Shot Learning for Attribute-Based Object Reference in Human-Robot Interaction
Authors:
Peng Gao,
Ahmed Jaafar,
Brian Reily,
Christopher Reardon,
Hao Zhang
Abstract:
Language-enabled robots have been widely studied over the past years to enable natural human-robot interaction and teaming in various real-world applications. Language-enabled robots must be able to comprehend referring expressions to identify a particular object from visual perception using a set of referring attributes extracted from natural language. However, visual observations of an object ma…
▽ More
Language-enabled robots have been widely studied over the past years to enable natural human-robot interaction and teaming in various real-world applications. Language-enabled robots must be able to comprehend referring expressions to identify a particular object from visual perception using a set of referring attributes extracted from natural language. However, visual observations of an object may not be available when it is referred to, and the number of objects and attributes may also be unbounded in open worlds. To address the challenges, we implement an attribute-based compositional zero-shot learning method that uses a list of attributes to perform referring expression comprehension in open worlds. We evaluate the approach on two datasets including the MIT-States and the Clothing 16K. The preliminary experimental results show that our implemented approach allows a robot to correctly identify the objects referred to by human commands.
△ Less
Submitted 21 December, 2023;
originally announced December 2023.
-
Attachment and antibiotic response of early-stage biofilms studied using resonant hyperspectral imaging
Authors:
Yue Wang,
Christopher P. Reardon,
Nicholas Read,
Stephen Thorpe,
Adrian Evans,
Neil Todd,
Marjan Van Der Woude,
Thomas F. Krauss
Abstract:
Many bacterial species readily develop biofilms that act as a protective matrix against external challenge, e.g. from antimicrobial treatment. Therefore, biofilms are often responsible for persistent and recurring infections. Established methods for studying biofilms are either destructive or they focus on the biofilm surface. A non-destructive method that is sensitive to the underside of the biof…
▽ More
Many bacterial species readily develop biofilms that act as a protective matrix against external challenge, e.g. from antimicrobial treatment. Therefore, biofilms are often responsible for persistent and recurring infections. Established methods for studying biofilms are either destructive or they focus on the biofilm surface. A non-destructive method that is sensitive to the underside of the biofilm is highly desirable, as it allows studying the penetration of antibiotics through the film. Here, we demonstrate that the high surface sensitivity of resonant hyperspectral imaging provides this capability. The method allows us to monitor the early stages of Escherichia coli biofilm formation, cell attachment and microcolony formation, in-situ and in real time. We study the response of the biofilm to a number of different antibiotics and verify our observations using confocal microscopy. Based on this ability to closely monitor the surface-bound cells, resonant hyperspectral imaging gives new insights into the antimicrobial resistance of biofilms.
△ Less
Submitted 7 September, 2020;
originally announced September 2020.
-
Simultaneous Learning from Human Pose and Object Cues for Real-Time Activity Recognition
Authors:
Brian Reily,
Qingzhao Zhu,
Christopher Reardon,
Hao Zhang
Abstract:
Real-time human activity recognition plays an essential role in real-world human-centered robotics applications, such as assisted living and human-robot collaboration. Although previous methods based on skeletal data to encode human poses showed promising results on real-time activity recognition, they lacked the capability to consider the context provided by objects within the scene and in use by…
▽ More
Real-time human activity recognition plays an essential role in real-world human-centered robotics applications, such as assisted living and human-robot collaboration. Although previous methods based on skeletal data to encode human poses showed promising results on real-time activity recognition, they lacked the capability to consider the context provided by objects within the scene and in use by the humans, which can provide a further discriminant between human activity categories. In this paper, we propose a novel approach to real-time human activity recognition, through simultaneously learning from observations of both human poses and objects involved in the human activity. We formulate human activity recognition as a joint optimization problem under a unified mathematical framework, which uses a regression-like loss function to integrate human pose and object cues and defines structured sparsity-inducing norms to identify discriminative body joints and object attributes. To evaluate our method, we perform extensive experiments on two benchmark datasets and a physical robot in a home assistance setting. Experimental results have shown that our method outperforms previous methods and obtains real-time performance for human activity recognition with a processing speed of 10^4 Hz.
△ Less
Submitted 26 March, 2020;
originally announced April 2020.
-
Representing Multi-Robot Structure through Multimodal Graph Embedding for the Selection of Robot Teams
Authors:
Brian Reily,
Christopher Reardon,
Hao Zhang
Abstract:
Multi-robot systems of increasing size and complexity are used to solve large-scale problems, such as area exploration and search and rescue. A key decision in human-robot teaming is dividing a multi-robot system into teams to address separate issues or to accomplish a task over a large area. In order to address the problem of selecting teams in a multi-robot system, we propose a new multimodal gr…
▽ More
Multi-robot systems of increasing size and complexity are used to solve large-scale problems, such as area exploration and search and rescue. A key decision in human-robot teaming is dividing a multi-robot system into teams to address separate issues or to accomplish a task over a large area. In order to address the problem of selecting teams in a multi-robot system, we propose a new multimodal graph embedding method to construct a unified representation that fuses multiple information modalities to describe and divide a multi-robot system. The relationship modalities are encoded as directed graphs that can encode asymmetrical relationships, which are embedded into a unified representation for each robot. Then, the constructed multimodal representation is used to determine teams based upon unsupervised learning. We perform experiments to evaluate our approach on expert-defined team formations, large-scale simulated multi-robot systems, and a system of physical robots. Experimental results show that our method successfully decides correct teams based on the multifaceted internal structures describing multi-robot systems, and outperforms baseline methods based upon only one mode of information, as well as other graph embedding-based division methods.
△ Less
Submitted 7 April, 2020; v1 submitted 26 March, 2020;
originally announced March 2020.
-
Enabling Intuitive Human-Robot Teaming Using Augmented Reality and Gesture Control
Authors:
Jason M. Gregory,
Christopher Reardon,
Kevin Lee,
Geoffrey White,
Ki Ng,
Caitlyn Sims
Abstract:
Human-robot teaming offers great potential because of the opportunities to combine strengths of heterogeneous agents. However, one of the critical challenges in realizing an effective human-robot team is efficient information exchange - both from the human to the robot as well as from the robot to the human. In this work, we present and analyze an augmented reality-enabled, gesture-based system th…
▽ More
Human-robot teaming offers great potential because of the opportunities to combine strengths of heterogeneous agents. However, one of the critical challenges in realizing an effective human-robot team is efficient information exchange - both from the human to the robot as well as from the robot to the human. In this work, we present and analyze an augmented reality-enabled, gesture-based system that supports intuitive human-robot teaming through improved information exchange. Our proposed system requires no external instrumentation aside from human-wearable devices and shows promise of real-world applicability for service-oriented missions. Additionally, we present preliminary results from a pilot study with human participants, and highlight lessons learned and open research questions that may help direct future development, fielding, and experimentation of autonomous HRI systems.
△ Less
Submitted 13 September, 2019;
originally announced September 2019.
-
Hybrid plasmonic waveguide coupling of photons from a single molecule
Authors:
Samuele Grandi,
Michael P. Nielsen,
Javier Cambiasso,
Sebastien Boissier,
Kyle D. Major,
Christopher Reardon,
Thomas F. Krauss,
Rupert F. Oulton,
E. A. Hinds,
Alex S. Clark
Abstract:
We demonstrate the emission of photons from a single molecule into a hybrid gap plasmon waveguide (HGPW). Crystals of anthracene, doped with dibenzoterrylene (DBT), are grown on top of the waveguides. We investigate a single DBT molecule coupled to the plasmonic region of one of the guides, and determine its in-plane orientation, excited state lifetime and saturation intensity. The molecule emits…
▽ More
We demonstrate the emission of photons from a single molecule into a hybrid gap plasmon waveguide (HGPW). Crystals of anthracene, doped with dibenzoterrylene (DBT), are grown on top of the waveguides. We investigate a single DBT molecule coupled to the plasmonic region of one of the guides, and determine its in-plane orientation, excited state lifetime and saturation intensity. The molecule emits light into the guide, which is remotely out-coupled by a grating. The second-order auto-correlation and cross-correlation functions show that the emitter is a single molecule and that the light emerging from the grating comes from that molecule. The coupling efficiency is found to be $β_{WG}=11.6(1.5)\%$. This type of structure is promising for building new functionality into quantum-photonic circuits, where localised regions of strong emitter-guide coupling can be interconnected by low-loss dielectric guides.
△ Less
Submitted 15 May, 2019;
originally announced May 2019.
-
Effect of Notch Structure on Magnetic Domain Movement in Planar Nanowires
Authors:
Hua Ling,
Junlin Wang,
Xianyang Lu,
Junran Zhang,
Li Chen,
Christopher Reardon,
Jason Zhang,
Yichuan Wang,
Yu Yan,
Jing Wu,
Yongbing Xu
Abstract:
We present the direct observation of magnetic domain motion in permalloy nanowires with notches using a wide-field Kerr microscopy technique. The domain wall motion can be modulated by the size and shape of the notch structure in the nanowires. It is demonstrated that the coercive fields can be tuned by modulating the notches. The experimental results are consistent with the micro-magnetic simulat…
▽ More
We present the direct observation of magnetic domain motion in permalloy nanowires with notches using a wide-field Kerr microscopy technique. The domain wall motion can be modulated by the size and shape of the notch structure in the nanowires. It is demonstrated that the coercive fields can be tuned by modulating the notches. The experimental results are consistent with the micro-magnetic simulation results. The relationship between the notch angle and the domain nuclease are also studied. This work is useful for the design and development of the notch-based spintronic devices.
△ Less
Submitted 23 November, 2017;
originally announced November 2017.
-
Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors
Authors:
Fei Han,
Xue Yang,
Christopher Reardon,
Yu Zhang,
Hao Zhang
Abstract:
Robot awareness of human actions is an essential research problem in robotics with many important real-world applications, including human-robot collaboration and teaming. Over the past few years, depth sensors have become a standard device widely used by intelligent robots for 3D perception, which can also offer human skeletal data in 3D space. Several methods based on skeletal data were designed…
▽ More
Robot awareness of human actions is an essential research problem in robotics with many important real-world applications, including human-robot collaboration and teaming. Over the past few years, depth sensors have become a standard device widely used by intelligent robots for 3D perception, which can also offer human skeletal data in 3D space. Several methods based on skeletal data were designed to enable robot awareness of human actions with satisfactory accuracy. However, previous methods treated all body parts and features equally important, without the capability to identify discriminative body parts and features. In this paper, we propose a novel simultaneous Feature And Body-part Learning (FABL) approach that simultaneously identifies discriminative body parts and features, and efficiently integrates all available information together to enable real-time robot awareness of human behaviors. We formulate FABL as a regression-like optimization problem with structured sparsity-inducing norms to model interrelationships of body parts and features. We also develop an optimization algorithm to solve the formulated problem, which possesses a theoretical guarantee to find the optimal solution. To evaluate FABL, three experiments were performed using public benchmark datasets, including the MSR Action3D and CAD-60 datasets, as well as a Baxter robot in practical assistive living applications. Experimental results show that our FABL approach obtains a high recognition accuracy with a processing speed of the order-of-magnitude of 10e4 Hz, which makes FABL a promising method to enable real-time robot awareness of human behaviors in practical robotics applications.
△ Less
Submitted 24 February, 2017;
originally announced February 2017.
-
Efficient silicon metasurfaces for visible light
Authors:
Zhenpeng Zhou,
Juntao Li,
Rongbin Su,
Beimeng Yao,
Hanlin Fang,
Kezheng Li,
Lidan Zhou,
Jin Liu,
Daan Stellinga,
Christopher P Reardon,
Thomas F Krauss,
Xuehua Wang
Abstract:
Dielectric metasurfaces require high refractive index contrast materials for optimum performance. This requirement imposes a severe restraint; devices have either been demonstrated at wavelengths of 700nm and above using high-index semiconductors such as silicon, or they use lower index dielectric materials such as TiO$_{2}$ or Si$_{3}$N$_{4}$ and operate in the visible wavelength regime. Here, we…
▽ More
Dielectric metasurfaces require high refractive index contrast materials for optimum performance. This requirement imposes a severe restraint; devices have either been demonstrated at wavelengths of 700nm and above using high-index semiconductors such as silicon, or they use lower index dielectric materials such as TiO$_{2}$ or Si$_{3}$N$_{4}$ and operate in the visible wavelength regime. Here, we show that the high refractive index of silicon can be exploited at wavelengths as short as 532 nm by demonstrating a silicon metasurface with a transmission efficiency of 47% at this wavelength. The metasurface consists of a graded array of silicon posts arranged in a square lattice on a quartz substrate. We show full 2π phase control and we experimentally demonstrate polarization-independent beam deflection at 532nm wavelength. The crystalline silicon is placed on a quartz substrate by a bespoke layer transfer technique and we note that an efficiency >70% may be achieved for a further optimized structure in the same material. Our results open a new way for realizing efficient metasurfaces based on silicon in the visible wavelength regime.
△ Less
Submitted 20 September, 2016;
originally announced September 2016.
-
Effect of lithographically-induced strain relaxation on the magnetic domain configuration in microfabricated epitaxially grown Fe81Ga19
Authors:
R. P. Beardsley,
D. E. Parkes,
J. Zemen,
S. Bowe,
K. W. Edmonds,
C. Reardon,
F. Maccherozzi,
I. Isakov,
P. A. Warburton,
R. P. Campion,
B. L. Gallagher,
S. A. Cavill,
A. W. Rushforth
Abstract:
We investigate the role of lithographically-induced strain relaxation in a micron-scaled device fabricated from epitaxial thin films of the magnetostrictive alloy Fe81Ga19. The strain relaxation due to lithographic patterning induces a magnetic anisotropy that competes with the magnetocrystalline and shape induced anisotropies to play a crucial role in stabilising a flux-closing domain pattern. We…
▽ More
We investigate the role of lithographically-induced strain relaxation in a micron-scaled device fabricated from epitaxial thin films of the magnetostrictive alloy Fe81Ga19. The strain relaxation due to lithographic patterning induces a magnetic anisotropy that competes with the magnetocrystalline and shape induced anisotropies to play a crucial role in stabilising a flux-closing domain pattern. We use magnetic imaging, micromagnetic calculations and linear elastic modelling to investigate a region close to the edges of an etched structure. This highly-strained edge region has a significant influence on the magnetic domain configuration due to an induced magnetic anisotropy resulting from the inverse magnetostriction effect. We investigate the competition between the strain-induced and shape-induced anisotropy energies, and the resultant stable domain configurations, as the width of the bar is reduced to the nanoscale range. Understanding this behaviour will be important when designing hybrid magneto-electric spintronic devices based on highly magnetostrictive materials.
△ Less
Submitted 10 February, 2017; v1 submitted 9 September, 2016;
originally announced September 2016.
-
Self-Reflective Risk-Aware Artificial Cognitive Modeling for Robot Response to Human Behaviors
Authors:
Fei Han,
Christopher Reardon,
Lynne E. Parker,
Hao Zhang
Abstract:
In order for cooperative robots ("co-robots") to respond to human behaviors accurately and efficiently in human-robot collaboration, interpretation of human actions, awareness of new situations, and appropriate decision making are all crucial abilities for co-robots. For this purpose, the human behaviors should be interpreted by co-robots in the same manner as human peers. To address this issue, a…
▽ More
In order for cooperative robots ("co-robots") to respond to human behaviors accurately and efficiently in human-robot collaboration, interpretation of human actions, awareness of new situations, and appropriate decision making are all crucial abilities for co-robots. For this purpose, the human behaviors should be interpreted by co-robots in the same manner as human peers. To address this issue, a novel interpretability indicator is introduced so that robot actions are appropriate to the current human behaviors. In addition, the complete consideration of all potential situations of a robot's environment is nearly impossible in real-world applications, making it difficult for the co-robot to act appropriately and safely in new scenarios. This is true even when the pretrained model is highly accurate in a known situation. For effective and safe teaming with humans, we introduce a new generalizability indicator that allows a co-robot to self-reflect and reason about when an observation falls outside the co-robot's learned model. Based on topic modeling and two novel indicators, we propose a new Self-reflective Risk-aware Artificial Cognitive (SRAC) model. The co-robots are able to consider action risks and identify new situations so that better decisions can be made. Experiments both using real-world datasets and on physical robots suggest that our SRAC model significantly outperforms the traditional methodology and enables better decision making in response to human activities.
△ Less
Submitted 16 May, 2016;
originally announced May 2016.
-
Integrated spatial multiplexing of heralded single photon sources
Authors:
Matthew J. Collins,
Chunle Xiong,
Isabella H. Rey,
Trung D. Vo,
Jiakun He,
Shayan Shahnia,
Christopher Reardon,
M. J. Steel,
Thomas F. Krauss,
Alex S. Clark,
Benjamin J. Eggleton
Abstract:
The non-deterministic nature of photon sources is a key limitation for single photon quantum processors. Spatial multiplexing overcomes this by enhancing the heralded single photon yield without enhancing the output noise. Here the intrinsic statistical limit of an individual source is surpassed by spatially multiplexing two monolithic silicon correlated photon pair sources, demonstrating a 62.4%…
▽ More
The non-deterministic nature of photon sources is a key limitation for single photon quantum processors. Spatial multiplexing overcomes this by enhancing the heralded single photon yield without enhancing the output noise. Here the intrinsic statistical limit of an individual source is surpassed by spatially multiplexing two monolithic silicon correlated photon pair sources, demonstrating a 62.4% increase in the heralded single photon output without an increase in unwanted multi-pair generation. We further demonstrate the scalability of this scheme by multiplexing photons generated in two waveguides pumped via an integrated coupler with a 63.1% increase in the heralded photon rate. This demonstration paves the way for a scalable architecture for multiplexing many photon sources in a compact integrated platform and achieving efficient two photon interference, required at the core of optical quantum computing and quantum communication protocols.
△ Less
Submitted 30 May, 2013;
originally announced May 2013.
-
Optimal Pacing for Running 400 m and 800 m Track Races
Authors:
James C. Reardon
Abstract:
Physicists seeking to understand complex biological systems often find it rewarding to create simple "toy models" that reproduce system behavior. Here a toy model is used to understand a puzzling phenomenon from the sport of track and field. Races are almost always won, and records set, in 400 m and 800 m running events by people who run the first half of the race faster than the second half, whic…
▽ More
Physicists seeking to understand complex biological systems often find it rewarding to create simple "toy models" that reproduce system behavior. Here a toy model is used to understand a puzzling phenomenon from the sport of track and field. Races are almost always won, and records set, in 400 m and 800 m running events by people who run the first half of the race faster than the second half, which is not true of shorter races, nor of longer. There is general agreement that performance in the 400 m and 800 m is limited somehow by the amount of anaerobic metabolism that can be tolerated in the working muscles in the legs. A toy model of anaerobic metabolism is presented, from which an optimal pacing strategy is analytically calculated via the Euler-Lagrange equation. This optimal strategy is then modified to account for the fact that the runner starts the race from rest; this modification is shown to result in the best possible outcome by use of an elementary variational technique that supplements what is found in undergraduate textbooks. The toy model reproduces the pacing strategies of elite 400 m and 800 m runners better than existing models do. The toy model also gives some insight into training strategies that improve performance.
△ Less
Submitted 2 April, 2012;
originally announced April 2012.
