Sub Sea Imaging

Client: Cathx Ocean

The role involved Research, Design and Development of a browser based image processing portal which interacts with a Microsoft dot net engine by means of a MQTT Internet of Things message broker.

One web based system, ‘Clarity’, to replace three existing desktop applications used by Cathx Ocean for Sub Sea Imaging:

1) Scene Config, which sets up the scene and camera, light and laser controls and sequencer.
2) Camera Monitor, which monitors the cameras live. And
3) SOLID, which is used for image and or laser processing.

Angular UI with Mosquitto MQTT Broker

Research

• Technology Nodes.io, Flutter or Angular?
• Review (SOLID) the existing desktop image and laser processing application.
• Email out a survey to customers and in-house Users, share with the team and take on board the feedback.
• Review ‘Camera Monitor’ and ‘Scene Config’ GUI that operate camera, light and laser controls and sequencer.
• Inspect the existing camera profiles Json file and do some quick analytics to get a better view of the data.
• Research the Engine (dot NET running Python scripts – GeoAPI, Extended Xml Serializer, Json.NET, OpenCV, MathNet Spatial, Nav, MQTTnet, Zio, logging and Meta Shape for navigation and License Spring to manage the distribution of licences.)
• Research sending lightweight MQTT IOT messages via web sockets.
• Define front-end stack (Angular, not Flutter, Material design, Node JS and, Express JS)

Design

• Host weekly workshops
• Showing design ideas, mock ups and prototypes.
• Generate Strategic Ideas including: current depth UI, demo hub on sub domain, working selection to allow User to select a short sample to focus on and transform before applying to all images.
• Demo UI components.
• Design screens for Camera Monitoring,  Scene Configuration and Workflow design.

Development

• Build a ‘Get Status’ proof of concept App that interacts with the Engine via MQTT to get and display the status of the engine. Commands are sent in Json format to the engine via MQTT pub sub model using ‘topics’ and ‘response topics’ which both the engine and UI subscribe to.
• Develop an Angular App which gets the available ‘workflows’ from the engine and displays a list to the User. (a workflow is a group of micro services which do various image and data transformations).
  • When the User clicks to select a ‘workflow’ the engine returns a list of variables related to that workflows micro services and builds them into a ‘stepper’ form User interface.
  • Every step in the stepper form is a micro service related to the workflow.
  • The prototype workflow was ‘offline undistort’, other workflows include: Camera Monitor (live undistort), Motion Correction, Sharpen, Object Detection, Make Mosaic Friendly, Mission Rules,  Select Mission Range, Select Mission Timeframe, Mission Logging and Mission Alerts.
  • The prototype workflow ‘offline undistort’ is a simple workflow which basically takes a cameras lens calibration file and executes a python script to correct for the visual distortion in an image taken with that lens.
  • The prototype workflow ‘offline undistort’ had only 4 exposed variables:
    1. The images source folder (the Cathx Ocean mission folder)
    2. The camera calibration file (this file is custom to each camera lens and supplied by Cathx Ocean)
    3. Select camera (which camera to process or select all cameras). And
    4. The images Output folder.
  • When the User has stepped through the stepper and the validation is all checked the User can then click ‘Run Workflow’.
  • The engine then goes to work creating the images; during this the UI is constantly querying the status of the engine to determine if the work is complete.
  • When the work is complete the User gets notified via Toast Message.
  • The engine then sends the ‘transformed’ images back to the User by base 64 encoding the images and sending via MQTT message.
  • The process of getting an image or laser file, transforming that data or just encoding it RAW and sending if via MQTT can be done to preview or view live data.