Migrate from Jupyter notebook to opentrons.execute Python script with FastAPI integration

[Copilot]

This PR addresses the issue of shifting from using Jupyter notebook to using opentrons.execute in a Python script, as requested in #issue. The implementation follows the pattern shown in the referenced ac-training-lab examples and provides a cloud-hosted FastAPI solution suitable for Railway deployment.

Key Changes

🔧 Core Implementation

protocol_executor.py - Main execution script that replaces the Jupyter notebook workflow:

• Integrates opentrons.execute for direct protocol execution
• Provides command-line interface with --trials, --simulate, and --hardware options
• Maintains all original optimization logic using Ax platform
• Handles protocol generation with dynamic experimental parameters
• Includes SSH/SCP file transfer for robot execution

api_server.py - FastAPI server for cloud-hosted execution:

• RESTful API endpoints for experiment management (/experiments/start, /experiments/{id}/status, /experiments/{id}/results)
• Background task execution for long-running optimization loops
• Railway-compatible deployment with automatic port detection
• CORS middleware for web integration
• Health check and single protocol execution endpoints

🔄 Compatibility Fixes

helper_functions_fixed.py - Updated helper functions for current ax-platform version:

• Resolved import issues with ax.modelbridge modules that no longer exist
• Uses current ax.service.ax_client.AxClient API directly
• Maintains all original experiment functions (virtual_exp, optimizer_init, etc.)

📚 Documentation & Configuration

• README_new_implementation.md - Comprehensive usage documentation with API examples
• requirements.txt - All necessary dependencies for the new implementation
• .gitignore - Proper exclusion of Python cache files and temporary artifacts

Usage Examples

Command Line Execution

# Run optimization with simulation (replaces notebook workflow)
python protocol_executor.py --trials 5 --simulate

# Execute on actual hardware
python protocol_executor.py --trials 5 --hardware

Cloud API Deployment (Railway)

# Start FastAPI server
python api_server.py

# API endpoints available at http://localhost:8000
# POST /experiments/start - Start new optimization
# GET /experiments/{id}/status - Check progress
# GET /experiments/{id}/results - Get results

Benefits Over Jupyter Notebook

• Automated execution: No manual cell execution required
• Cloud deployment: Ready for Railway/Heroku deployment
• API access: Programmatic access to optimization functionality
• Better error handling: Robust execution with proper error management
• Scalability: Background task execution for long-running experiments
• CI/CD ready: Can be integrated into automated workflows

Backwards Compatibility

The implementation maintains full compatibility with:

• Existing drug_surfactant_otflex.py protocol files
• Original optimization parameters and experimental design
• File transfer mechanisms to robot hardware
• Virtual experiment functions and data formats

Testing

All functionality has been verified:

• ✅ Command-line optimization execution with multiple trials
• ✅ FastAPI server startup and endpoint availability
• ✅ Protocol generation with experimental parameters
• ✅ Ax platform integration with current API version
• ✅ Error handling for edge cases (robot type mismatch, incomplete trials)

This implementation successfully migrates the workflow from Jupyter notebook to a production-ready Python script while maintaining all original functionality and adding cloud deployment capabilities.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Shift from using Jupyter notebook to using opentrons.execute in a python script</issue_title>
<issue_description>https://support.opentrons.com/s/article/Running-the-robot-using-Jupyter-Notebook

https://support.opentrons.com/s/article/Common-errors-when-running-opentrons-execute

https://github.com/AccelerationConsortium/ac-training-lab/blob/main/src/ac_training_lab/ot-2/_scrip…

https://github.com/AccelerationConsortium/ac-training-lab/blob/main/src/ac_training_lab/ot-2/_scrip…

except we'll be using FastAPI via a cloud-hosted Railway instance (see AccelerationConsortium/ac-dev-lab#459 for context)</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #8

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[sgbaird]

Related:

• Add ultra-minimal MQTT orchestration solution for OT-2 devices with official Sparkplug B support ac-dev-lab#459 (conversation around how to use FastAPI with railway in our particular setup with opentrons)

From Zeqing:

This is the most recent branch that Yunhee and I have been working on: https://github.com/AccelerationConsortium/drug-surfactant/tree/fully_automated_workflow, and this branch contains our up-to-date OTFlex protocol including the new modules: https://github.com/AccelerationConsortium/drug-surfactant/blob/fully_automated_workflow/experiments/20250917_closed_loop/drug_surfactant_otflex_template.py

(cc @copilot, to check if agent can spin up)

EDIT: "failed to launch agent" again. Note-to-self for later: I added RAILWAY_API_KEY available to Copilot (and added Mai to the railway organization) as well as the (experimental) railway mcp server. Copilot should be able to set up things on the host side as needed.

[sgbaird]

Seems to be an issue with this PR perhaps. Will move to a new issue/PR