Paul Thienphrapa

From Paul Thienphrapa
Jump to navigation Jump to search

I am a scientist and engineer in computer vision and robotics at Atlas5D. Previously I was a research scientist at Philips and a PhD student in computer science at Johns Hopkins University. Mepaul is a neo-nickname my mom created, perhaps to give my full name some balance—an experimentation mindset that drives me to learn, everything from soldering to neural networks and beyond.

Russell Taylor (left) observes as Paul Thienphrapa (middle) and Tutkun Sen (right) manipulate a minimally invasive surgical tool.
Photo by Will Kirk / Johns Hopkins University

Contact Info
first at (work)
first.last at (personal)
pault at (historical)


My professional interests include surgical robotics and navigation via medical images. I built a smart, dexterous robot with 3D ultrasound guidance for beating heart surgery as my PhD thesis. I am broadly interested in innovative components, systems, and techniques emerging from technical and clinical insights. The end-to-end engineering of complete, real-time systems has been a highlight of my work.

Meanwhile, these folks and communities have been the highlights of my growth:

  • Russell H. Taylor, Johns Hopkins University, international celebrity pioneer of medical robotics, who showed me that being the best is fleeting, but persistently improving is real
  • Peter Kazanzides, Johns Hopkins University, inventor of the first-ever active surgical robot, who helped me realize that a system is way more than just the sum of its parts
  • Aleksandra Popovic, Philips Research, my industry advisor and mentor, who developed my love of churning ambiguous theory into practical realities that help people
  • LCSR – The Laboratories for Computational Sensing and Robotics
    • CIIS – Computer Integrated Interventional Systems Lab
    • SMARTS – Sensing, Manipulation and Real-Time Systems Lab
    • CISST – NSF Engineering Research Center preceding the LCSR
  • Countless colleagues across organizations large and small, time zones, geographies, and backgrounds


  • PhD, Computer Science | Surgical Robotics
Johns Hopkins University
Snake robot for cardiac surgery with 3D ultrasound vision (with R. Taylor, P. Kazanzides, A. Popovic)
  • MSE, Computer Science | Embedded Systems
Johns Hopkins University
Open source robot control platform over high speed networks (with P. Kazanzides)
  • MS, Electrical Engineering | Embedded Systems (pending thesis)
California State University, Los Angeles
Reconfigurable, fault-tolerant embedded system for space telescope control (with C. Liu)
  • BS, Electrical & Computer Engineering
California Institute of Technology
Hackerman Hall / JHU


  • Head of Computer Vision (present)
    ^ Principal Computer Vision Scientist (1 year)
    Atlas5D, Cambridge, MA
    • Develop unique machine learning techniques for contact-free health sensing, bringing offline prototypes to international commercial deployment in 4 months
    • Lead development of deep learning algorithms for fast, competitive computer vision capabilities with live upgrades
    • Build cloud infrastructure to automatically train on live data and scale novel AI models to commodity edge platforms
  • Senior Robotics Research Scientist (2 years)
    ^ Robotics Research Scientist (4 years)
    Philips Innovation Center, Cambridge, MA
    • Design surgical navigation systems with robotics, computer vision, medical image guidance
    • Collaborate on diverse teams with technical experts, clinicians, and business units [2 awards]
    • Engineer prototypes, report to stakeholders on analysis, competitive landscape, and IP [20+]
    • Publish experimental enhancements in visualization, workflow, and UX [peer-reviewed and presented at top-flight conferences ICRA and MICCAI]
  • Robotics and Computer Vision Engineer (4 summers + 1 winter as intern)
    Philips Research, Briarcliff Manor, NY
    • Design image guidance algorithms for surgical robots via ultrasound and endoscopy
    • Develop software for closed-loop robotics, visual servo control, tracking, and navigation
  • Research Assistant (5 years as PhD student; see Grad Projects)
    Johns Hopkins University (ERC-CISST/LCSR), Baltimore, MD
    • Engineer sophisticated, 11-DoF, miniature snake robots for beating heart surgery
    • Devise and analyze smart strategies for safe robot control in the heart using 3D ultrasound
    • Architect the original framework (electronics, FPGA/firmware, software/API) leading to the da Vinci Research Kit (dvrk), an open source motion controller now used as a standard platform for surgical robotics research across 40+ institutions worldwide
    • Mentor students, evaluate projects, and review assignments in surgical systems courses
  • Software Engineer (1 summer as co-op, 1.5 years as full-time contractor)
    Sierra Monolithics (Broadband Wireless and Optical Circuits), Redondo Beach, CA
    • Software lead in a rapidly growing firm, automating and streamlining processes
    • Automate ASIC test protocols from engineering through production cycles
    • Optimize tests, strategically triggering signal generators and oscilloscopes
    • Streamline data collection and reporting via new software infrastructure
    • Collaborate with OEMs (VxWorks) to integrate devices and create intuitive UIs
  • Research Assistant (2.5 years as master's student; see Prior Projects)
    California State University, Los Angeles, SPACE Engineering Lab
    • Develop multiprocessor architecture prototypes using parallel programming and IPC
    • Project 1: Fault-tolerant embedded computing to control a segmented space telescope
    • Project 2: High volume and robust astronomical image processing and transmission
  • Software Engineer (1 summer as intern, 1+ years as freelance contractor)
    Unified Dispatch, Altadena, CA
    • Design and develop a mobile app for GPS navigation in taxis and shuttles
    • Integrate speech recognition for hands-free use, databases for fare processing
    • Demonstrated and well received at an industry trade show

Note: Most of the above experiences *overlap* with adjacent ones, for those keeping track.


Additional projects can be found on ResearchGate.

Grad Projects

Robotic retrieval of foreign bodies from a beating heart under 3D ultrasound guidance

We develop a minimally invasive surgical robot to extract foreign objects that find their way into the heart. The system uses streaming 3D ultrasound images (known in the business as transesophageal echocardiograms or TEE) to track a foreign body as it flies around the chamber, then strategically guides a dexterous robotic end effector to capture the erratic target.

Scalable IEEE 1394 (FireWire)-based motion controller

While designing robots with many degrees of freedom (such as the snake robots above and below), we ran into physical limitations in the number of joints/axes we could integrate and furthermore the cabling complexity became unwieldy. So we developed a real-time motion controller with an architecture that could scale over a high speed network—IEEE 1394.

This effort turned out to be broadly useful for developing high-dof robots—the electronics design, FPGA firmware, low-level drivers, and basic software tools formed the basis for the da Vinci Research Kit (dvrk), an open source motion controller for da Vinci surgical systems that is now used as a standard platform for surgical robotics research worldwide.

Snake Robot

This snake robot is a telerobotic surgical assistant designed for minimally invasive surgery of the throat and upper airways. It consists of two miniature manipulators that operate through a single entry port, a laryngoscope. The goal is to help surgeons perform tasks in confined spaces by enhancing their distal dexterity.

Prior Projects

Robotic guidance of an uncalibrated endoscope in beating heart surgery

The rotated and small fields of view of endoscopes used in minimally invasive surgery can lead to counterintuitive images that hamper the workflow. We counter this using visual servoing, wherein the endoscope is robotically maneuvered based on features extracted from the video stream. The methods are designed to be efficient yet robust against imperfect camera calibration.

Real-time, fault tolerant embedded computing for decentralized control of a segmented telescope

To improve sensitivity to faint signals, the James Webb Space Telescope features an extra large reflector that is collapsible in order to fit into contemporary launch vehicles. When deployed, the reflector segments must be actively and precisely controlled to act as a single mirror in a dynamic disturbance environment. We explore different parallel processor embedded computing system designs to achieve real-time, fault tolerant control of a complex structure with many degrees of freedom.

Optimal and robust astronomical image processing and transmission

The James Webb Space Telescope will generate vast quantities of high quality astronomical images. To facilitate widespread dissemination, we investigate different image processing and transmission techniques to extract significant information content and prioritize data packets, thereby maximizing delivered image quality under channel constraints.

Miscellaneous memorable projects

  • Standalone MP3 player: Developed an x86-based embedded system including microprocessor, controllers, DRAM, storage, etc.; drew schematics, timing diagrams; wire-wrapped circuits; programmed in assembly—from the infamous EE 52.
  • Operating systems: Wrote kernel-level software on Linux including task queues, scheduler policies, and a filesystem driver loaded as a kernel module—from the lesser known but equally painful CS 134a (now defunct—too painful perhaps).
  • Intelligent ICU: Integrated multiple bedside monitoring devices in an intensive care setting by bringing disjoint datastreams into one computer, allowing for intelligent and automated assessment of patient condition. Our group won best course project, in case you like that sort of thing (historical project link).
  • Other fond memories: Projects involving computer architecture, analog/digital circuit design, PCB and VLSI layout, all orders of coding: network, FPGA, µC/µP, embedded, mobile (pre-iPhone though), device driver, UI, scripting, even a bit of web and database. (I don't claim 'current' expertise in all of these things...)



  • Philips Emergency Response Team—On call for emergency assistance, maintain training in First Aid CPR
  • LA Inner City Games—Build and run a non-profit computer lab; mentor youths with educational software
  • Tutor and mentor local neighborhood kids in K–12 in reading, math, and science
  • STEM outreach—Help organize, run, and judge youth robotic competitions; lead lab tours; host visiting students; run open house demos and talks

Scientific Reviewer - Regularly referee submissions to premier journals and conferences in robotics, computer vision, and medical image guidance (winning Best Reviewer for MICCAI 2020 and IPCAI 2017)

  • ICRA – IEEE International Conference on Robotics and Automation
  • BioRob – IEEE International Conference on Biomedical Robotics and Biomechatronics
  • MICCAI – Medical Image Computing and Computer Assisted Interventions
  • IJCARS – International Journal for Computer Assisted Radiology and Surgery
  • MVAP – Machine Vision and Applications
  • JMRR – Journal of Medical Robotics Research
  • IPCAI – Information Processing in Computer Assisted Interventions



  • Intercollegiate (NCAA Division III) soccer, track & field (4 years)
  • Interscholastic (Los Angeles City Section) soccer, track & field, cross-country (4, 3, 2 years)
  • Intramural basketball, soccer, track & field (great exercise w/multiple championships)


... and things like it.


  • IPCAI Best Reviewer Award
  • MICCAI Outstanding Reviewer Award
  • Philips Accelerator Entrepreneurial Team Award
  • Philips Innovation X-Challenge 3rd Prize

Fellowships – Graduate

  • Philips Research (4 years)
  • Transportation Research Board (travel)
  • CSULA Engineering Ben Levine
  • CSULA Graduate Equity (2 years)

Scholarships – Undergraduate

  • Robert C. Byrd Honors (State of CA, 4 years)
  • Los Angeles Inner City Games (city-wide)
  • Los Angeles Unified School District Employees
  • John Wooley Memorial (student-athlete)


  • Finalist, Los Angeles City Section triple jump
  • Captain, varsity soccer and cross-country teams
  • All-Conference, NCAA Div. III 4x100m and 4x400m relays
  • Coaches Award, scholastic track & field and collegiate soccer
  • Athlete of the Week, scholastic and collegiate


  • Selected for NASA/JPL Planetary Science Summer School
  • Mock Trial team ranked #1 in Los Angeles County (at one point)
  • Best Project: Intelligent ICU (600.446: Computer Integrated Surgery)
  • Al Bundy award for over-representation of high school sports accomplishments
Photos by Ian Shapiro (1-2) and Ely Ruiz (3)



  1. P. Thienphrapa, T. Bydlon, A. Chen, P. Vagdargi, N. Varble, D. Stanton, and A. Popovic, “Interactive endoscopy: A next-generation, streamlined user interface for lung surgery navigation,” in Int. Conf. on Medical Image Computing and Computer Assisted Intervention (MICCAI), Shenzhen, China, Oct 2019.
  2. P. Thienphrapa, P. Vagdargi, A. Chen, and D. Stanton, “User centric device registration for streamlined workflows in surgical navigation systems,” in IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 2641–2647, Montreal, Canada, May 2019.
  3. P. Thienphrapa, B. Mory, A. Panse, V. Pai Raikar, A. Torjesen, D. Schulman, M. Flexman, and A. Popovic, “Basic principles for integrating next generation technologies into surgical workflows,” in ICRA Next Generation Surgery Workshop, May 2019.
  4. P. Thienphrapa, B. Mory, A. Panse, V. Pai Raikar, A. Torjesen, and D. Schulman, “Usability in surgical technology: Finding research questions in practical problems,” in ICRA Industry Forum, May 2019.
  5. P. Thienphrapa, T. Bydlon, A. Chen, and A. Popovic, “Evaluation of surface feature persistence during lung surgery,” in Biomedical Engineering Society (BMES) Annual Meeting, Atlanta, GA, Oct 2018.
  6. P. Vagdargi, P. Thienphrapa, A. Chen, and T. Bydlon, “Towards augmented reality for lung surgery,” in National Center for Image Guided Therapy Workshop, Boston, MA, Oct 2018.
  7. P. Thienphrapa, A. Popovic, and R. Taylor, “Guidance of a high dexterity robot under 3D ultrasound for minimally invasive retrieval of foreign bodies from a beating heart,” in IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 4869–4874, Hong Kong, China, May 2014.
  8. P. Thienphrapa, A. Popovic, and R. Taylor, “3D ultrasound-guided retrieval of foreign bodies from a beating heart using a dexterous surgical robot,” in Hamlyn Symposium on Medical Robotics, London, UK, Jun 2013.
  9. P. Thienphrapa, B. Ramachandran, H. Elhawary, A. Popovic, and R. Taylor, “Intraoperative analysis of locations for 3D ultrasound-guided capture of foreign bodies from a beating heart,” in Hamlyn Symposium on Medical Robotics, London, UK, Jul 2012.
  10. P. Thienphrapa, B. Ramachandran, R. Taylor, and A. Popovic, “A system for 3D ultrasound-guided robotic retrieval of foreign bodies from a beating heart,” in IEEE RAS/EMBS Int. Conf. on Biomedical Robotics and Biomechatronics (BioRob), pp. 743–748, Rome, Italy, Jun 2012.
  11. P. Thienphrapa, B. Ramachandran, H. Elhawary, R. Taylor, and A. Popovic, “Multiple capture locations for 3D ultrasound-guided robotic retrieval of moving bodies from a beating heart,” in SPIE 8316, Medical Imaging 2012: Image-Guided Procedures, Robotic Interventions, and Modeling, pp. 831619, San Diego, CA, Feb 2012.
  12. B. Ramachandran, P. Thienphrapa, A. Jain, and A. Popovic, “Tracking using 3D ultrasound for guiding cardiac interventions,” in Int. Conf. on Biomedical Engineering (ICBME), Manipal, India, Dec 2011.
  13. P. Thienphrapa and P. Kazanzides, “Design of a scalable real-time robot controller and application to a dexterous manipulator,” in IEEE Int. Conf. on Robotics and Biomimetics (RoBio), pp. 2295–2300, Phuket, Thailand, Dec 2011.
  14. P. Thienphrapa, H. Elhawary, B. Ramachandran, D. Stanton, and A. Popovic, “Tracking and characterization of fragments in a beating heart using 3D ultrasound for interventional guidance,” in Int. Conf. on Medical Image Computing and Computer Assisted Intervention (MICCAI), vol. 6891, pp. 211–218, Toronto, Canada, Sep 2011.
  15. A. Popovic and P. Thienphrapa, “An approach to robotic guidance of an uncalibrated endoscope in beating heart surgery,” in IEEE RAS/EMBS Int. Conf. on Biomedical Robotics and Biomechatronics (BioRob), pp. 106–113, Tokyo, Japan, Sep 2010. [DOI]
  16. P. Thienphrapa and P. Kazanzides, “A scalable system for real-time control of dexterous surgical robots,” in IEEE Int. Conf. on Technologies for Practical Robot Applications (TePRA), pp. 16–22, Boston, MA, Nov 2009.
  17. P. Thienphrapa and P. Kazanzides, “A distributed I/O low-level controller for highly-dexterous snake robots,” in IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 9–12, Baltimore, MD, Nov 2008.
  18. P. Kazanzides and P. Thienphrapa, “Centralized processing and distributed I/O for robot control,” in IEEE Int. Conf. on Technologies for Practical Robot Applications (TePRA), pp. 84–88, Boston, MA, Nov 2008.
  19. P. Thienphrapa and J. Dong, “A robust transmission system for astronomical images over error-prone links,” in SPIE Newsroom, Dec 2006. [DOI]
  20. P. Thienphrapa, H. Boussalis, C. Liu, K. Rad, and J. Dong, “Implementation of a robust transmission system for astronomical images over error-prone links,” in SPIE 6391, Multimedia Systems and Applications IX, pp. 63910G, Boston, MA, Oct 2006. [DOI]
  21. P. Thienphrapa, H. Boussalis, C. Liu, K. Rad, J. Dong, “Content-based retransmission with error concealment for astronomical images,” in SPIE 6015, Multimedia Systems and Applications VIII, pp. 60150J, Boston, MA, Oct 2005.
  22. S. Fallorina, P. Thienphrapa, R. Luna, V. Khuong, H. Boussalis, C. Liu, J. Dong, K. Rad, and W. Ho, “A fault-tolerant distributed data flow architecture for real-time decentralized control,” in Int. Conf. on Informatics in Control, Automation and Robotics (ICINCO), pp. 109–115, Barcelona, Spain, Sep 2005.
  23. H. Boussalis, C. Liu, K. Rad, J. Dong, S. Fallorina, P. Thienphrapa, and J. Roberts, “Integrated embedded architectures and parallel algorithms for a decentralized control system,” in IEEE Int. Sym. on Intelligent Control (ISIC), Mediterranean Conf. on Control and Automation, pp. 1567–1572, Limassol, Cyprus, Jun 2005.
  24. J. Roberts, H. Boussalis, C. Liu, J. Dong, K. Rad, P. Thienphrapa, Z. Purnajo, and S. Fallorina, “Efficient real-time parallel signal processing for decentralized control using group-pipelined scheduling,” in Information Systems: New Generations Conference (ISNG), Las Vegas, NV, Nov 2004.
  25. S. Fallorina, H. Boussalis, C. Liu, K. Rad, J. Dong, D. Nasser, and P. Thienphrapa, “A generic pipelined task scheduling algorithm for fault-tolerant decentralized control of a segmented telescope testbed,” in ASME Computers and Information in Engineering Conference (CIE), vol. 4, pp. 493–500, Salt Lake City, UT, Sep 2004.
  26. P. Thienphrapa, S. Fallorina, Z. Purnajo, E. Prince, H. Boussalis, C. Liu, K. Rad, J. Dong, and Y. Zhao, “A generalized fault-tolerant pipelined task scheduling for decentralized control of large segmented systems,” in Int. Conf. on Computing, Communications, and Control Technologies (CCCT), Austin, TX, Aug 2004.



  1. T. Bydlon, P. Thienphrapa, and M. Flexman, “Medical navigation system using shape-sensing device and method of operation thereof,” EP3565482B1, Mar 2021.
  2. A. Popovic, P. Thienphrapa, and G. Toporek, “Image-based fusion of endoscopic image and ultrasound images,” EP3463032B1, Dec 2020.
  3. X. He, A. Popovic, M. Flexman, P. Thienphrapa, D. Noonan, R. Kroon, and A. Reinstein, “Shape sensing for orthopedic navigation,” JP6713987B2, Jun 2020.
  4. G. Cole, P. Thienphrapa, M. Flexman, D. Noonan, and N. Kahya, “System and method for tracking and determining characteristics of inflatable medical instruments using fiber-optical realshape data,” EP3313498B1, JP6739454B2, May 2020.
  5. P. Thienphrapa, B. Ramachandran, and A. Popovic, “Automatic online registration between a robot and images,” US9984437B2, EP2745268B1, RU2624107C2, CN103797512B, JP6491476B2, May 2018.
  6. A. Popovic and P. Thienphrapa, “Uncalibrated visual servoing using real-time velocity optimization,” US8934003B2, US9205564B2, EP2521507B1, JP5814938B2, CN102791214B, Jan 2015.


  1. A. Popovic and P. Thienphrapa, “Robotic control of an oblique endoscope for FOV images,” WO2012001549A1, Priority Jun 2010.
  2. M. Flexman, A. Reinstein, X. He, P. Thienphrapa, D. Dijkkamp, and D. Noonan, “Triggering with optical shape sensing fiber,” WO2016051302A1, Priority Sep 2014.
  3. P. Thienphrapa and X. He, “System and method for registering a structure using fiber-optical realshape data,” WO2016207163A1, Priority Jun 2015.
  4. P. Thienphrapa, B. Ramachandran, A. Reinstein, and D. Stanton, “Registration system for medical navigation and method of operation thereof,” WO2017115201A1, Priority Dec 2015.
  5. P. Thienphrapa and A. Popovic, “Registration of a surgical image acquisition device using contour signatures,” WO2017114828A1, Priority Dec 2015.
  6. P. Thienphrapa, B. Ramachandran, M. Flexman, and N. Kahya, “Medical navigation system employing optical position sensing and method of operation thereof,” WO2018002109A1, Priority Jun 2016.
  7. M. Flexman, M. Marell, and P. Thienphrapa, “Balloon catheter comprising shape sensing optical fibers,” WO2018011158A1, Priority Jul 2016.
  8. T. Bydlon, P. Thienphrapa, M. Flexman, A. Panse, A. Patriciu, and S. Kyne, “Shape sensing of multiple over-the-wire devices,” WO2018096491A1, Priority Nov 2016.
  9. T. Bydlon, A. Ekin, P. Thienphrapa, W. Van Den Boomen, M. Flexman, M. Van Der Mark, and A. Popovic, “Systems and methods for determining the length of a non-shape-sensed interventional device with a shape-sensed guidewire and determining a state of the guidewire with respect to an interventional device,” WO2018104162A1, Priority Dec 2016.
  10. P. Thienphrapa, T. Bydlon, M. Flexman, A. Patriciu, A. Panse, and S. Kyne, “Systems and methods for determining the position of a non-shape-sensed guidewire with a shape-sensed catheter and for visualizing the guidewire,” WO2018108993A1, Priority Dec 2016.
  11. M. Flexman, P. Thienphrapa, T. Bydlon, A. Patriciu, and A. Panse, “OSS guiding and monitoring systems, controllers and methods,” WO2018172237A1, Priority Mar 2017.
  12. A. Ekin, M. Flexman, P. Thienphrapa, and W. Van Den Boomen, “OSS foreshortening detection systems,” WO2018178248A1, Priority Mar 2017.
  13. P. Thienphrapa, M. Flexman, T. Bydlon, A. Popovic, M. Balicki, G. Toporek, and A. Patriciu, “Autonomous X-ray control for robotic navigation,” WO2019092225A1, Priority Nov 2017.
  14. P. Thienphrapa, M. Flexman, T. Bydlon, A. Popovic, M. Balicki, G. Toporek, and A. Patriciu, “Robotic tool control,” WO2019092261A1, Priority Nov 2017.
  15. P. Thienphrapa, P. Cathier, A. Panse, M. Flexman, T. Bydlon, and N. Kahya, “Torsional deployment detection of a vascular therapy device,” WO2019121889A1, Priority Dec 2017.
  16. N. Kahya, O. Nempont, P. Thienphrapa, T. Bydlon, P. Cathier, M. Flexman, and R. Florent, “Animated position display of an OSS interventional device,” WO2019134898A1, Priority Jan 2018.
  17. T. Bydlon, P. Thienphrapa, and A. Torjesen, “Registering optical shape sensing device with three-dimensional representation of region of interest,” WO2020002176A2, Priority Jun 2018.
  18. T. Bydlon, P. Thienphrapa, and A. Chen, “Dynamic interventional three-dimensional model deformation,” WO2020182997A1, Priority Mar 2019.
  19. T. Bydlon, P. Thienphrapa, and A. Chen, “Intraoperative imaging-based surgical navigation,” WO2020234409A1, Priority May 2019.
  20. P. Thienphrapa, T. Bydlon, A. Chen, P. Vagdargi, and W. McNamara, “Interactive endoscopy for intraoperative virtual annotation in VATS and minimally invasive surgery,” WO2021048326A1, Priority Sep 2019.