State of the art virtual reality requires low latency, on the order of single-digit milliseconds to present a scene to an operator using immersive tracking devices. In conventional teleoperation, the operator's movements are used to directly control a remote camera, requiring a fast response time to avoid simulator sickness, an approach not feasible for interplanetary robotic control with limited bandwidth, high latency communications. The proposed work develops a hybrid architecture to present a model of a coarse virtual world to an operator, while the real imagery slowly refines the presentation as bandwidth allows. An online physics engine allows an actuation command in the virtual world to be faithfully executed in the real world without direct operator feedback. The end goal is a smooth, visually realistic operator interface that allows for remote operation of an exploration vehicle in a blended world scene of virtual and actual data.