I am a Data Engineer who brings a system-level engineering mindset to building scalable data infrastructure.
My journey is unique: I spent over 4 years as a Mechanical Design Engineer at Dyson, solving complex physical optimization and system integration challenges. I have translating 'systems-first' engineering rigor —optimizing for constraints, efficiency, and reliability—directly into Distributed Data Systems. This pivot has been a deliberate, built on a foundation of continuous self-study and hands-on projects, and capped off with a Master's in Computer Science. I don’t just write pipelines; I engineer fault-tolerant, high-throughput systems.
I thrive on architecting and optimizing the backend. I'm actively seeking full-time Data Engineer.
I specialize in the intersection of Backend Engineering and Data Infrastructure. I am driven by the challenge of "building it right"—creating clean, performant architectures that handle massive scale without technical debt.
- Modern Data Architecture: I enjoy architecting streaming and batch solutions with open-sourced tools.
- Backend & Mathematical Logic: I love coding systems with strong logical or mathematical foundations at their core. I like applying concepts like multithreading, multiprocessing, and vectorization to optimize performance. I'm currently exploring the bridge between high-level and high-performance languages, such as Python Rust bindings.
- Creative Integration: I excel at integrating, fine-tuning, and customizing existing tools, models, and techniques to solve a specific problem.
- Programming Languages: Python, SQL, Rust (WIP)
- Data Engineering: Apache Kafka, Apache Spark, Apache Flink
- Databases, Format & Storage: Apache Iceberg, Apache Paimon, Clickhouse (OLAP), MinIO (Object)
- Visualization & BI: Apache Superset, Grafana
- Containerization & Orchestration: Docker
- Observability & Monitoring: Grafana, Prometheus, Tempo, Loki
- Libraries: Pandas, NumPy, Scikit-learn, Pytorch
- Concepts: Event Streaming and Processing, Data Pipelines (ETL/ELT), System Design, Data Modeling
I am dedicated to continuous improvement and lifelong learning. If you have any feedback, suggestions for enhancement, or innovative ideas for any of my projects, please don't hesitate to open an issue or reach out directly. Your insights are invaluable as I strive to refine my skills and code. Nevertheless, I am always open to discussing technology, data system, optimization, or interesting opportunities.
- LinkedIn:
www.linkedin.com/in/chan-chen-lam - Email:
[email protected]