Currently buildingprivate
PCB inspection with a phone, right in the browser: point the camera at a board, it is recognized, the placement data is laid over it, and each component can be zoomed into and checked. Camera frames stay on the device.
Private project. Code and data are not public. This page shows occasional previews of the progress, not a product. Questions or test access: send a request.
How it works
- RecognizeThe board is found in the camera image by its features (OpenCV in WebAssembly) and its pose is estimated.
- OverlayComponent positions and outlines from the placement data are drawn over the live image and follow the board.
- Deep ZoomZoom into a single component while tracking keeps the view stable, even with a shaky hand.
- InspectEach component gets a result: fine, conspicuous, or not checkable from this view.
Current focus
Right now I am building FastAOI: the stabilized Deep Zoom mode combined with a fast automatic optical inspection, so a whole board is evaluated automatically and every component is judged by a model of its own.
- The board is split into cells that are zoomed into and captured one after another.
- Several frames per component are combined, so one blurry frame does not decide.
- Components are judged against references, later by an own trained model.
- A quick swipe review confirms or rejects results and calibrates the thresholds.
- Top and bottom side of the board are told apart automatically (draft).
- An autopilot runs photos, videos and scans through the app, measures how well each box sits, rates the crops and tunes thresholds, references and dataset itself, but only after a regression has passed.
Next steps
The latest phone run: about 19 poses a second at 30 camera frames, the overlay visible in three of four frames, and the anchors work on the device: they correct the pose by 1.5 to 14 px while tracking. What is left is a slight stutter. About every two seconds an anchor search runs for 0.6 to 0.9 s, tracking takes longer meanwhile and camera frames are dropped.
- A cheaper feature search for anchors during tracking: the pose is already known there, so fewer features or a faster detector than AKAZE are enough.
- Refining the anchor without the extra load on the tracking.
Both then go through the video regression and a run on the device again. Rarer anchors, fewer flow points, smaller frames, a search window for the anchor and other thread splits have already been tried and ruled out.
Possible next step
A possible combination with AIO Process Optimization, my public sandbox for AOI models: its Simple-Sim pipeline generates synthetic datasets of components and defects, trains classifiers on them and ranks them in an arena. Such synthetic data could train DeepVision’s own component model for better recognition, before enough real board images exist. An idea under consideration, not a fixed plan.
AIO Process OptimizationPrinciples
Image processing runs on the device. Tuning with a local LLM only ever sees numbers, never images.
A pose held in the wrong place is not a success. False alarms and misses are counted separately, and “not checkable” is a result of its own.
It points people at what to look at. The decision stays with the person doing the inspection.
Where it stands
- Board recognition and overlayWorking
- Deep Zoom with stabilized trackingWorking
- FastAOI with Deep ZoomBuilt, measured on recordings
- Automatic top/bottom detectionDraft
- Own component modelPlanned
- Synthetic training data from Simple-SimUnder consideration
Deep Zoom with stabilized tracking works. FastAOI gives verdicts on real boards only once the camera is calibrated.