Utilization of FPGA for Onboard Inference of Landmark Localization in CNN-Based Spacecraft Pose Estimation

  title={Utilization of FPGA for Onboard Inference of Landmark Localization in CNN-Based Spacecraft Pose Estimation},
  author={Kiruki Cosmas and Asami Kenichi},
In the recent past, research on the utilization of deep learning algorithms for space applications has been widespread. One of the areas where such algorithms are gaining attention is in spacecraft pose estimation, which is a fundamental requirement in many spacecraft rendezvous and navigation operations. Nevertheless, the application of such algorithms in space operations faces unique challenges compared to application in terrestrial operations. In the latter, they are facilitated by powerful… 
Resource-Constrained Onboard Inference of 3D Object Detection and Localisation in Point Clouds Targeting Self-Driving Applications
This paper investigates the use of a deep learning-based method in edge devices for onboard real-time inference that is power-effective and low in terms of space-constrained demand, and proposes a hybrid FPGA-CPU board as an effective onboard inference solution.
Deep Learning-based Spacecraft Relative Navigation Methods: A Survey
Environmental Sound Recognition on Embedded Systems: From FPGAs to TPUs
This work proposes and adjusts several CNN-based sound classifiers to be embedded on hardware accelerators and demonstrates the maturity of the existing tools and how FPGAs can be exploited to outperform TPUs.
Optimize Neural Network Implementation on FPGA
The results show that the artificial neural network can be implemented in FPGA to achieve hardware implementation of face recognition system.
Towards On-Board Hyperspectral Satellite Image Segmentation: Understanding Robustness of Deep Learning through Simulating Acquisition Conditions
This paper proposes a set of simulation scenarios that reflect a range of atmospheric conditions and noise contamination that may ultimately happen on-board an imaging satellite, and verifies their impact on the generalization capabilities of spectral and spectral-spatial convolutional neural networks for hyperspectral image segmentation.
Study on an Estimation Method of On-orbit Single Event Upset Rate Based on Historical Data
Single event effect is a main reliability problem for satellite in space. It is a difficult task to estimate the single event upset(SEU) rate of SRAM-based FPGA in different satellite orbits. In this
Juxtaposing Vivado Design Flows in Batch Mode
This paper presents a comparative study considering all available synthesis and implementation strategies of Xilinx’s Vivado Design Suite, and tries to pinpoint key characteristics that may be effective in view of future endeavors.
Elastic computing self-organizing for artificial intelligence space exploration
A methodology useful for designing and controlling elastic computing self-organizing for artificial intelligence space exploration by using elastic infrastructure model and the terminology of graph dynamical systems to be able to capture a broad variety of processes taking place on self- Organizing networks.


Assistive Relative Pose Estimation for On-orbit Assembly using Convolutional Neural Networks
A convolutional neural network is leveraged to uniquely determine the translation and rotation of an object of interest relative to the camera in order to assist object tracker used in on space assembly tasks where only feature based method is always not sufficient.
Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks
The proposed architecture is scalable to spacecraft of different structural and physical properties as well as robust to the dynamic illumination conditions of space and can be used to augment the current state-of-the-art monocular vision-based pose estimation techniques used in spaceborne applications.
A robust pose estimator based on a Convolutional Neural Network that can directly estimate the three-dimensional locations of keypoints in real-time and is validated experimentally using realistic images obtained by a hardware simulator that can take pictures under the space environment.
Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA
This paper proposes Angel-Eye, a programmable and flexible CNN accelerator architecture, together with data quantization strategy and compilation tool, which achieves similar performance and delivers up to better energy efficiency than peer FPGA implementation on the same platform.
Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement
This work proposes an approach to estimate the 6DOF pose of a satellite, relative to a canonical pose, from a single image, by predicting the coordinates of a set of landmarks in the input image, associating the landmarks to their corresponding 3D points on an a priori reconstructed 3D model, then solving for the object pose using non-linear optimisation.
Pose estimation of an uncooperative spacecraft from actual space imagery
The aim is to estimate the pose of a passive space resident object using its known three-dimensional model and single low-resolution two-dimensional images collected on-board the active spacecraft.
Going Deeper with Embedded FPGA Platform for Convolutional Neural Network
This paper presents an in-depth analysis of state-of-the-art CNN models and shows that Convolutional layers are computational-centric and Fully-Connected layers are memory-centric, and proposes a CNN accelerator design on embedded FPGA for Image-Net large-scale image classification.
Onboard Processing With Hybrid and Reconfigurable Computing on Small Satellites
It is demonstrated how system designers can exploit hybrid and reconfigurable computing on SmallSats to harness these advantages for a variety of purposes, and several recent missions by NASA and industry that feature these principles and technologies are highlighted.
[DL] A Survey of FPGA-based Neural Network Inference Accelerators
An investigation from software to hardware, from circuit level to system level is carried out to complete analysis of FPGA-based neural network inference accelerator design and serves as a guide to future work.
An Efficient Task Assignment Framework to Accelerate DPU-Based Convolutional Neural Network Inference on FPGAs
A high performance task assignment framework built upon Xilinx hybrid CPU-FPGA MPSoC devices and integrated with optimizations to maximize performance on DPU-based CNN acceleration platform is presented.