SplatOverflow: Asynchronous Hardware Troubleshooting

Amritansh Kwatra , Cornell Tech Tobias Weinberg 1, Ilan Mandel 1, Ritik Batra 1, Peter He , Cornell University Francois Guimbretiere 2, Thijs Roumen 1

preprint video publication code

Figure 1: A SplatOverflow Scene Figure 1. (a) A SplatOverflow scene, comprising a 3D Gaussian Splat aligned and registered onto a digital CAD model, acting as a boundary object to facilitate asynchronous troubleshooting tasks for hardware. A local end-user can query technical documentation and past issues associated with the hardware by simply clicking on the parts in the SplatOverflow scene. If no solution is found, they can use SplatOverflow to request and receive help asynchronously from a remote maintainer. (b) A remote maintainer uses SplatOverflow to explore a local end-user's hardware and author instructions for them to follow. (c) A local end-user views the maintainer's instructions rendered as an overlay onto their hardware in their workspace and executes the specified action.

Abstract

As tools for designing and manufacturing hardware become more accessible, smaller producers can develop and distribute novel hardware. However, processes for supporting end-user hardware troubleshooting or routine maintenance aren't well defined. As a result, providing technical support for hardware remains ad-hoc and challenging to scale. Inspired by patterns that helped scale software troubleshooting, we propose a workflow for asynchronous hardware troubleshooting: SplatOverflow. SplatOverflow creates a novel boundary object, the SplatOverflow scene, that users reference to communicate about hardware. A scene comprises a 3D Gaussian Splat 3D Gaussian Splatting [Kerbl et al. 2023] is a view-interpolation and rasterization technique that renders a 3D scene as a collection of Gaussian distributions. of the user's hardware registered onto the hardware’s CAD model. The splat captures the current state of the hardware, and the registered CAD model acts as a referential anchor for troubleshooting instructions. With SplatOverflow, remote maintainers can directly address issues and author instructions in the user’s workspace. Workflows containing multiple instructions can easily be shared between users and recontextualized in new environments. In this paper, we describe the design of SplatOverflow, the workflows it enables, and its utility to different kinds of users. We also validate that non-experts can use SplatOverflow to troubleshoot common problems with a 3D printer in a usability study.

Demonstrative Examples

We demonstrate SplatOverflow with a variety of hardware examples, including a pick-and-place machine, a 3D printer, and an open-source e-reader.

Figure 2. A SplatOverflow scene of a pick-and-place machine, the Opulo Lumen V3 registered onto it's CAD model rendered as a yellow wireframe. For a detailed example of troubleshooting a problem on this pick-and-place using SplatOverflow, please see our video or read the Walkthrough section of our paper.
Figure 3. A SplatOverflow scene of a 3D printer, the Prusa MK3S+ registered onto it's CAD model rendered as a yellow wireframe. Our evaluation examined whether non-experts could use SplatOverflow to troubleshoot common problems with this printer. For more details, please read the Evaluation section of our paper.
Figure 4: A SplatOverflow Scene of the OpenBook e-reader rendering PCB traces overlaid onto the board Figure 4. A still of a SplatOverflow scene depicting the PCB for an open-source e-reader, the Open Book registered onto it's CAD model. In (a) electrical traces from the PCB layout are overlaid onto the board as a yellow wireframe. In (b) electrical pads are highlighted.

Source Code & Documentation

The source code and accompanying technical documentation of our implementation will be available shortly under a CC BY-SA license. If you'd like access before the public release, please reach out using the email address below.

BibTex


@misc{kwatra2024splatoverflowasynchronoushardwaretroubleshooting,
    title={SplatOverflow: Asynchronous Hardware Troubleshooting}, 
    author={Amritansh Kwatra and Tobias Wienberg and Ilan Mandel and Ritik Batra 
            and Peter He and Francois Guimbretiere and Thijs Roumen},
    year={2024},
    eprint={2411.02332},
    archivePrefix={arXiv},
    primaryClass={cs.HC},
    url={https://arxiv.org/abs/2411.02332}, 
}

Acknowledgements

We would like to thank the Digital Life Initiative at Cornell Tech for supporting this work through a doctoral fellowship. We would also like to thank Joey Castillo, Frank Bu and Stephen Hawes for taking part in preliminary discussions that helped motivate this work.

Contact

If you have questions about this work, contact Amrit Kwatra: ak2244 at cornell dot edu.