Add Ocean Insight QE Pro spectrometer driver#87
Open
dccote wants to merge 4 commits into
Open
Conversation
The QE Pro is a scientific-grade CCD spectrometer (Hamamatsu S7031-1006, 1024 active pixels, 8 ms - 60 min integration, TEC-cooled) that is not supported by oceaninsight.py. The reason it cannot just be added as a USB2000/USB4000 sibling is that the QE Pro speaks an entirely different wire protocol — Ocean Binary Protocol (OBP) — used by all modern Ocean Insight spectrometers (STS, FX, HDX, ...) but not by the older USB-series models in oceaninsight.py. OBP is a fixed-layout binary protocol: 44-byte header, optional payload, 16-byte checksum block, 4-byte footer (0xC5C4C3C2). Every transaction carries a 32-bit opcode, optional 16-byte immediate-data slot for small operands, and per-message ACK/NACK flags. The full spec is captured in manuals/QE-Pro-Data-Sheet.pdf (pages 21-29 for the protocol, pages 30+ for the message catalog). Changes in hardwarelibrary/spectrometers/: - obp.py: OBPMessage class that encodes outgoing requests and decodes responses. Validates start bytes, footer, NACK, exception, and error number. Kept transport-agnostic (no USB), so unit tests can verify the byte layout directly against the two worked examples in the data sheet (Set Integration Time = 0x00110010 on page 26, Get Buffered Spectrum = 0x00100928 on page 27). - qepro.py: QEPro(Spectrometer) driver using PyUSB on endpoint 1 (EP1 OUT/IN per the data sheet). Implements the Spectrometer interface (getSpectrum, setIntegrationTime, getIntegrationTime) plus getSerialNumber and wavelength-calibration retrieval. getSpectrum follows the Abort -> Clear -> Acquire -> Get sequence the data sheet prescribes for software-triggered acquisitions — the QE Pro buffers spectra, so the naive "request and read" pattern used by USB2000 returns stale data. DebugQEPro(QEPro) overrides sendOBP() to dispatch on message type and return synthesized responses, so the entire driver code path (wavelength polynomial, spectrum parsing, integration-time bounds checking) is exercised without hardware. Uses classIdVendor=0xFFFF / classIdProduct=0xFFF1 to avoid collisions with real USB devices and with the existing debug devices in this codebase (linearmotion 0xFFFD, labjack 0xFFFB). - __init__.py: export QEPro and DebugQEPro. Per CLAUDE.md, the new code subclasses spectrometers/base.py:Spectrometer directly rather than oceaninsight.OISpectrometer, and does not adopt the half-finished commands-dict pattern. Changes in hardwarelibrary/tests/: - testOBP.py: 16 tests including byte-by-byte verification against the two worked examples from the data sheet. - testQEPro.py: 13 TestDebugQEPro tests that always run, plus 3 TestQEPro tests that skip when no hardware is connected (per the PR #65/#66 pattern). Changes in hardwarelibrary/manuals/: - QE-Pro-Data-Sheet.pdf: vendor reference, alongside the existing USB2000/USB4000/HR2000+ data sheets in this directory. Known limitations: - Spectrometer.any() in base.py filters by the regex r'\.(USB.*?)\W' on the class name, so it will not pick up QEPro. Instantiate the class directly: qepro = QEPro(); qepro.initializeDevice(). Fixing the regex is a separate concern that also affects StellarNet. - The driver has only been validated against the protocol spec and the in-process DebugQEPro. Real-hardware validation (USB endpoint discovery, bulk-read chunking across the 4272-byte spectrum response) still needs to happen at the bench. - Out of scope for this commit: TEC control (0x00420010/11), trigger modes (0x00110110), lamp enable (0x00110410), nonlinearity correction, GPIO/strobe, irradiance calibration. All opcodes are in the data sheet and can be added via sendOBP() when needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PyHardwareLibrary already wraps pyusb in hardwarelibrary/communication/usbport.py:USBPort, which handles device lookup, configuration, interface selection, endpoint resolution, and buffered reads. The first cut of qepro.py bypassed that wrapper and talked to usb.core/usb.util directly — same hardware-access pattern the project has already standardized away from in IntegraDevice, SutterDevice, CoboltDevice, etc. Changes in hardwarelibrary/spectrometers/qepro.py: - Drop the usb.core / usb.util imports. - doInitializeDevice creates a USBPort(idVendor, idProduct, interfaceNumber=0, defaultEndPoints=(0, 1)) and opens it. The endpoint pair is exposed as the class attribute defaultEndPoints so a QE Pro unit that lists its descriptors in a different order can be supported with a one-line subclass override. - doShutdownDevice calls port.close() and clears self.port. - sendOBP writes via self.port.writeData() instead of touching the endpoint object. - _readMessage shrinks from 13 lines of bytearray accumulation to 4 lines: USBPort.readData() blocks until the requested length arrives, so we just ask for the 44-byte header, parse bytes_remaining, then ask for the rest. - portTimeoutMs (default 5000 ms) replaces the separate read/write timeouts; USBPort uses a single defaultTimeout for both directions. DebugQEPro is unaffected by the refactor — it overrides doInitializeDevice/doShutdownDevice/sendOBP and never touches a port. Verified: 29 passed, 3 skipped (TestQEPro hardware-gated as expected). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
0x2457:0x4004), the first device in this codebase to speak Ocean Binary Protocol (OBP). All modern Ocean Insight spectrometers (STS, FX, HDX, etc.) use the same protocol, soobp.pyis intended to be reused as more devices are added.spectrometers/base.py:Spectrometerdirectly (perCLAUDE.md), notOISpectrometer— the USB-series wire protocol inoceaninsight.pyis unrelated.DebugQEProcompanion (VID0xFFFF, PID0xFFF1) that overridessendOBP()to return synthesized OBP responses, so the entire driver code path is exercisable without hardware.Why a separate
obp.pyThe OBP wire format is fixed-layout binary (44-byte header + payload + 16-byte checksum + 4-byte footer, opcodes, ACK/NACK flags). Splitting it from
qepro.pykeeps it transport-agnostic and lets the unit tests verify the byte layout directly against the two worked examples in the QE Pro Data Sheet (Set Integration Time, page 26; Get Buffered Spectrum, page 27) without needing pyusb mocks.Files
hardwarelibrary/spectrometers/obp.py—OBPMessageencode/decode, validation, error codeshardwarelibrary/spectrometers/qepro.py—QEPro(Spectrometer)+DebugQEPro(QEPro)hardwarelibrary/spectrometers/__init__.py— exportQEPro,DebugQEProhardwarelibrary/tests/testOBP.py— 16 tests (incl. byte-by-byte data sheet verification)hardwarelibrary/tests/testQEPro.py— 13TestDebugQEPro(always run) + 3TestQEPro(skip when no hardware)hardwarelibrary/manuals/QE-Pro-Data-Sheet.pdf— vendor referenceKnown limitations
Spectrometer.any()inbase.py:108-112filters by the regexr'\.(USB.*?)\W'on the class name and will not pick upQEPro. Instantiate directly:QEPro().initializeDevice(). Same pre-existing issue affectsStellarNet; fixing the regex is out of scope here.DebugQEPro. Real-hardware bring-up (USB endpoint discovery, bulk-read chunking across the 4272-byte spectrum response, the ~7 s post-reset delay) still needs to happen at the bench.0x00420010/11), trigger modes (0x00110110), lamp enable (0x00110410), nonlinearity correction, GPIO/strobe, irradiance calibration. All opcodes are documented in the data sheet and can be added viasendOBP()when needed.Test plan
python3 -m pytest hardwarelibrary/tests/testOBP.py -v— 16 passedpython3 -m pytest hardwarelibrary/tests/testQEPro.py -v— 13 passed, 3 skipped (no hardware connected)TestQEProend-to-end —initializeDevice,getSerialNumber,setIntegrationTime,getSpectrumAbort -> Clear -> Acquire -> Getsequence produces a fresh spectrum (not stale buffered data)🤖 Generated with Claude Code