The Advanced Exploration Systems (AES) Logistics Reduction (LR) project Radio-frequency identification (RFID) Enabled Autonomous Logistics Management (REALM) task focuses on the subset of autonomous logistics management functions pertaining to automated localization and inventory of all physical assets pertaining to, or within, a vehicle utilizing RFID technologies. REALM technology can provide detailed data to enable autonomous operations such as automated crew procedure generation and robotic interaction with logistics and deep space habitats; this is especially of value where communication delays with Earth drive the need for self-reliance. The REALM project will conduct a series of ISS technology demonstrations. The first ISS demonstration, REALM-1, started in February 2017 and was completed at the end of FY19 when it was transitioned to the ISS program for sustaining operations. The second ISS demonstration, REALM-2, started late 2019 and will continue for at least one year.
The problem of locating all mission items within and around a vehicle is complicated by many factors, including the desire to rely only on passive tags, restrictions on RF transmit power, layered storage of logistics, the challenging RF scattering environment of vehicles, and metallic storage enclosures. To address this complex problem, associated RFID technologies are partitioned into three classes:
Dense Zone technologies
Sparse Zone technologies
Complex Event Processing
Dense Zone technologies pertain to enclosures with conductive, or shielded, boundaries and an integrated RFID reader to interrogate the items contained therein. Sparse zone technologies address all areas exclusive of the dense zones, including the open areas of a habitat module in addition to crevices, for example, behind a rack. These technologies include fixed-zone readers, steered-beam antenna readers, and mobile readers such as robotic elements, crew-held readers, or crew-worn readers. With both dense and sparse zones, guaranteed real-time, on-demand reads are not possible, so “smart” applications, e.g., Complex Event Processing (CEP), are required to infer item locations based on context from the sparse and dense zone technologies.
Mission details might drive a different combination of these three technologies. Therefore, in addition to maturing these individual technology areas, the LR REALM team will learn which combinations of technologies are best suited for specific missions. For example, dense zone technologies can be made highly accurate but entail greater mass compared to sparse zone technologies. Sparse zone technologies typically cover greater volume per reader, but are more apt to miss tags because they cover a larger area. They still require readers, cables, and antennas to accomplish their function. The operational intelligence provided by CEP can likely be traded for the size, weight, and power associated with dense and sparse zone technologies, but the extent, and specific implementation, remain as knowledge gaps to be addressed by this effort.
The REALM task is divided into four sub-technology projects: REALM-1, -2, -3, -6DoF, and REALM-RFID Sensing
REALM-1 infrastructure will be developed and evaluated on ISS, with RFID hatch readers and antennas deployed in ISS Node 1, U.S. Laboratory, and Node 2. A ground-based CEP center will receive data from the ISS hatch readers and will provide operational intelligence that infers item locations. This effort is in collaboration with the ISS program payloads office and the ISS vehicle office, both of which provide cost sharing for development. In FY15, manufacturing of the hatch readers, known as EMBER (EMBEdded RFID Reader), began, along with resident software development. In parallel, the CEP center was established, and the CEP team, including a university partner, began tailoring prior CEP work to NASA’s REALM goals. The REALM Test Bed was utilized for testing CEP concepts of operation prior to the processing of ISS REALM-1 data in succeeding years. In FY16, the hatch readers, antennas, and RF cables were developed. REALM-1 was launched December 2016. Testing, evaluation, and advancement of the CEP will continue using the REALM Test Bed in advance of REALM-1 data downlinked from ISS. FY17 through FY19 were devoted to the 24-month ISS technology demonstration of REALM-1. Multiple cycles of visiting vehicles, and the subsequent loading, off-loading, and translation of cargo through ISS will provide for thorough REALM-1 assessment. During this time, the CEP software will reside in a ground system and utilize the ISS REAM-1 data with crew activity data, inventory surveys, and imagery to improve the CEP location algorithms and evaluate the effectiveness of the hatch reader locations and ability to assess tagged item locations in non-REALM instrumented nodes. The REALM-1 system was considered sufficiently matured in FY19 and was transitioned to an operational ISS system midway through FY19, so that ISS became responsible for sustaining engineering of flight and ground REALM-1 assets.
With the REALM-1 on-orbit system having been fully transitioned from Payload status to System on ISS, the CEP/machine learning approaches remain a central focus going forward. This task serves four primary purposes. First, the largest improvements to date in CEP localization occurred with machine learning classifier approaches in FY20. Further trials on two different machine learning (ML) algorithms are planned in FY21 to further extract value from this technology as deeper knowledge of both the ML tools and the problem space evolve. Second, new context from REALM-2 and REALM-3 will be folded into the CEP, a critical step in understanding the value, impacts, and interdependencies of fixed reader systems, mobile reader systems, and stowage reader systems in ALM. Third, the REALM Analog at JSC, which is a horizontal cylindrical habitat test bed similar to HALO in dimensions, will be employed to understand the impacts of habitat shell geometry and interior stowage regions and furnishings on localization accuracy. Fourth and finally, the REALM Analog will be used to better understand and improve the ISS CEP performance, given the ability to better control experiments with an increased truth data size.
AES is continuing CEP evolution. Targets for CEP evolution include extensions to permit one-half-rack level localization (i.e., bay and quadrant – overhead, deck, port, starboard). Conventional deterministic algorithms were initially the most promising in terms of location accuracy; however, late in FY19 two machine learning/artificial intelligence (AI) approaches began to show considerable promise.
Other CEP work in FY21 includes continuation of tag motion inferences and mining of crew procedures. Identifying tag motion enables other capabilities such as determining items that are being transferred to a visiting vehicle – either as intended or otherwise. Mining of crew procedures allows for additional CEP context in estimating item locations and enables auto-alerts when REALM detects that logistics required in next-day’s procedures are not where IMS indicates. Crew procedure mining efforts were initiated in FY20 with results and conclusions anticipated in FY21.
In addition, other ISS experiments at various stages of maturity are being developed with anticipated use of the REALM-1 infrastructure. These include the REALM-2 and REALM-3 systems, described in more detail below, the 6-Degrees of Freedom (DoF) Wireless Hybrid Identification and Sensing Platform for Equipment Recovery (WHISPER), crew clothing tracking experiment to close an exploration knowledge gap, and a wearable RFID-sensing platform. High level REALM-1 activities are shown in the timeline below.
REALM-2 is an AES LR RFID interrogator payload on the Space Technology Mission Directorate (STMD) Next Generation Free-Flyer (NGFF), aka Astrobee, located inside the ISS, that will take RFID "snapshots" during cargo movement and refine item localization. In FY15, REALM-2 and Astrobee project teams initiated discussions and identified a preliminary payload architecture and preliminary interfaces. In FY16, the REALM-2 activity initiated formal interface development with the Astrobee project. The REALM-2 task also initiated development of flight software that will reside in the mobile reader. In FY17 and FY18, REALM-2 matured portions of the flight hardware design and software. The REALM-2 flight hardware was fabricated in FY19. Ground testing with an integrated flight-like Astrobee configuration was completed. In FY19, all REALM-2 flight hardware and safety certification were completed. The REALM-2 flight system was delivered for a November FY20 launch. End-to-end Astrobee ground integration tests occurred in October with the REALM-2 team at JSC controlling the Astrobee certification unit/REALM-2 reader on the granite table at ARC. Crew installation of REALM-2 was completed in January 2021, with commissioning and checkout completed and further experiments occurring throughout FY21 and FY22.
Significant early benefits of the Astrobee and REALM-2 systems are anticipated in the provision of new training data for REALM-1, as well as new vantage points provided by the REALM-2 system for the CEP location engines. Although early commissioning and experiments will occur in the ISS Japanese Pressurized Module (JPM) module, REALM-2 experiments in other modules, including Node3, will occur as Astrobee commissioning extends to those modules. Node3 is of particular importance as it is not currently instrumented by REALM-1, and the adjoining Permanent Multi-Purpose Module (PMM) is the primary ISS logistics module.
Following initial commissioning in the JPM module, plans are being made to introduce the REALM-2 payload into NOD3 so that it gains RFID visibility into the PMM, which contains the highest density of stowage on ISS. This would provide needed data to update related REALM key performance parameters.
The REALM-2 technology is being contemplated for Gateway Phase II following the introduction of intravehicular robotics. The ability to extend the coverage of a fixed-reader system could provide significant value in dispensable elements such as the Logistics module, which will be jettisoned at the end of each mission.
REALM-3 is a dense zone reader enclosure in the form of “smart” drawers, cargo bags, and work bags, where a large number of RFID-tagged items occupy a relatively small volume. Open reader systems such as REALM-1 do not typically handle regions such as this with high read accuracy, either due to the high tag density or metallic racks/enclosures. NOD1O4 and NOD1S4 are both zero-g stowage racks packed densely with food BOBs (Bulk Overwrap Bags) that have a metal foil wrap. The “bungee jail” stow area at the time of this RFID query also contained BOBs. Those three regions constitute 85% of the missed tags, owing to both the dense packing and RF opaque wrap. The dense zone technologies are expected to greatly reduce the number of tags that are missed and are thus likely to be a significant element of a deep-space-habitat ALM system.
In FY21, flight development of the ISS Payload “RFID-Sensing” will be completed for launch. “RFID-Sensing” comprises Smart Stow and the Drawer Monitor System. Both of these technologies, were conceived to overcome the aforementioned challenges with dense packing and/or items with metal or liquid content. The Smart Stow system comprises RFID signals routed to antennas embedded in a stowage enclosure. While highly effective, Smart Stow requires too much instrumentation to replicate throughout all racks on a vehicle. The Drawer Monitor System complements Smart Stow and entails much lower mass, so it is more amenable to wide proliferation throughout a vehicle, with Smart Stow allotted only to stowage enclosures of very frequent usage. The Drawer Monitor System is based on RFID sensor technology and hence is addressed in that section below.
In FY20, due to the significant potential benefits for ISS and Gateway, HYDRA has been an early focal point and was rapidly advanced from TRL 3 to TRL 6. HYDRA is a self-multiplexing antenna system that harvests a very small fraction of the RF energy at each stage to power an embedded ultra-low power microcontroller and RF switch. In this way, it avoids the additional power and control lines required for traditional multiplexers.
A primary attribute of HYDRA is that it increases performance by allowing proliferation of antennas without the increase of infrastructure that has been associated with prior art. However, in the context of space applications, in which low mass is critical, efforts are required to produce lower mass HYDRA nodes and antennas. To this end, the REALM team is collaborating with the ISM team for innovative approaches to manufacturing of HYDRA nodes, HYDRA antennas, and eventually, integrated HYDRA antenna nodes. Figure 15 highlights the targets for the collaboration, which includes In Situ Manufacturing (ISM) advanced manufacturing techniques applied to the HYDRA nodes, HYDRA antennas, and eventually, a conceived integrated HYDRA antenna nodes. In addition to mass reduction - remote reconfigurability, flex circuit implementations, and a hybrid antenna/junction node will be considered. Several near-term (i.e., FY21) flight adoptions are possible, depending on funding availability.
REALM-6DoF (6 degrees of freedom)
The REALM team is collaborating with Advanced Systems and Technologies (AS&T) to advance an ultra-precise WHISPER (Wireless Hybrid Identification and Sensing Platform for Equipment Recovery) RFID tracking system that will be compatible with the REALM-1 infrastructure. The AS&T WHISPER technology is being developed under a Small Business Innovative Research (SBIR) award. Their new tag and tracking system enhancement permits theoretical tracking accuracies at or below the 10-centimeter level and also permits orientation tracking. In FY20, the REALM team and AS&T will assess flight readiness and advance towards flight designs of the key hardware components. AS&T will develop the second version projector, the IWPv2, which will be the final flight form factor. Select components will be evaluated for space thermal and ionizing radiation environments. Specific con-ops for an ISS TD will be developed, reader synchronization plan will developed, and a crew-assisted calibration process will be developed. In addition, hardware prototypes will be delivered by AS&T to the REALM team to permit performance evaluation and begin system software integration.
In FY20, the REALM team and AS&T began advancing towards flight designs of the key hardware components. The initial priority of the effort will be advancing the system level TRL in order to fully demonstrate the technology to customer stakeholders. Several major strides in the system optics were achieved.
In FY21, AS&T will continue advancing toward a system-level demonstration. Previous demonstrations were limited to sensor tracking within a plane. The FY21 planned demonstration will be virtual with the system demonstrated throughout a habitat-like structure at the AS&T facility in California.
The REALM team is leveraging RFID integrated circuits (ICs) that offer serial interfaces in addition to the more conventional over-the-air radiated interface to an RFID tag. The serial interface permits attachment of a microcontroller and low-power sensors such that the resulting tag is capable of returning sensor data in addition to the typical code that uniquely identifies the tag. In FY20, the REALM team is applying this technology in the form of RFID tags that monitor drawer state; i.e., “open” or “closed.” These drawer sensor tags will be read by the REALM-1 readers. In addition, the REALM-1 system will read tagged items until they are placed inside of a drawer and cease to be “seen.” Data pulled from the drawer sensor tags will relay changes in drawer state and the time of those events. The CEP system will use that sensor data, in addition to the data read from the sought tag, and infer whether the tag has been moved into a drawer.
Also, in 2018, the REALM-1 software was modified in the REALM ground analog to retrieve data from the tag user memory banks, in preparation for WHISPER tags and wearable CO2 sensing tags. New tag antennas were conceived and modeled for support of WHISPER and RFID-sensing functions. AES LR awarded a SBIR phase IIX to AS&T to advance the technology toward flight
In FY19, the REALM team worked with AS&T to formulate a flight experiment plan, and with potential robotic partners to formulate applications relating to robotic precursor missions in which the 6DoF tag system enables robotic grappling and deployment of logistics.
In FY20, due to the potential benefits for ISS and Gateway, Hyper-Distributed RFID Antenna system (HYDRA) has been an early focal point.
In FY20, the team developed flight versions of the drawer tags for subsequent delivery for launch and ISS technology demonstration in FY21. In parallel, the team will advance other sensor types for the RFID tag platform, such as carbon dioxide.
In FY21, the REALM team will complete fabrication and assembly of the flight Drawer Monitor System tags. Software modifications necessary to extract the sensor data from the tags may also be made. The team will launch the tags with the Smart Stow system in FY21 or FY22.More »
The REALM technology has the potential to dramatically reduce crew time expended on general inventory management and searching for lost items. The REALM-1 ISS technology demonstration has had several successful finds of lost items that provide the initial validation of crew time savings. Moreover, assured localization of assets can enable heterogeneous packing to optimize volume efficiency rather than crew-time efficiency.
Currently, foam is used to package items less densely in order to facilitate crew access to items. REALM can allow rapid location of items in densely packed Cargo Transfer Bags (CTBs) that could reduce foam usage in logistics packaging by up to 50%. The reduction in foam volume will provide increased habitation volume in logistics vehicles and deep space habitats. For robotic precursor missions, REALM technology can enable machine interaction with logistics, including packing and assembly functions in advance of crew arrival. In particular, the REALM-6DoF in combination with REALM-1, has a host of other potential applications, including that of robotic navigation aid. It thus has the potential to satisfy the ALM technology roadmap gap of a so-called 6-degree-of-freedom tag system.More »
|Organizations Performing Work||Role||Type||Location|
|Johnson Space Center (JSC)||Lead Organization||NASA Center||Houston, Texas|
|Ames Research Center (ARC)||Supporting Organization||NASA Center||Moffett Field, California|
|Marshall Space Flight Center (MSFC)||Supporting Organization||NASA Center||Huntsville, Alabama|
|NASA Headquarters (HQ)||Supporting Organization||NASA Center||Washington, District of Columbia|
|University of Kentucky||Supporting Organization||Academia||Lexington, Kentucky|
|University of Massachusetts (UMASS)||Supporting Organization||Academia||Amherst, Massachusetts|
|Altius Space Machines, Inc.||Industry||Broomfield, Colorado|
|Baylor College of Medicine||Academia||Houston, Texas|