Research Engineer- Post Training (Co-op/Intern)

About Themis Intelligence

Themis

Intelligence builds the Utility Knowledge Base (UKB) and

Human-Guided Intelligence (HGI) platforms, redefining how utilities

operate. Our systems transform complex operational data into clear,

high-confidence decisions. We design software that empowers grid

professionals to think faster, act decisively, and operate with

precision in critical environments. Every product we ship is built

for real-world performance : reliable, observable, and secure from

day one. We are an AI-first organization, continuously applying

modern tools and workflows to improve how we plan, execute, and

deliver outcomes. About the Role

We’re looking

for a co-op student for a 12-month duration who’s curious about how

LLM agents behave in real-world applications—especially when it

comes to hallucination, grounding, and evaluation. You’ll be

working on the post training lifecycle of Themis Agents (e.g.,

knowledge base assistants, alarm summarizers, and SCADA-aware

chatbots), ensuring that models are not only powerful but also

accurate and trustworthy. You’ll contribute to designing and

running evaluation frameworks, comparing retrieval strategies,

testing prompt chains, and analyzing model behavior across a range

of tasks. This is a hands-on, research-oriented role ideal for

someone with AI-related coursework and a passion for building

better, safer AI systems. In this role, you

will

Evaluation &

Grounding

Evaluate Themis Agents for accuracy,

factual consistency, hallucination, and tool correctness. Analyze

grounding failures—when models “go off-script” from retrieved

knowledge or internal documents. Score and compare outputs across

tasks like Q&A, summarization, and event reasoning. Experiment

with prompt templates, few-shot examples, and retrieval settings.

Compare vector store search performance using embedding models,

chunking strategies, and context window variations. Run A / B tests

across model versions and prompt chains. Tooling &

Automation

Build or extend evaluation

pipelines in Python and frameworks like LangChain, OpenAI API, or

Transformers. Visualize and organize test results using tools like

Streamlit, pandas, or Dash. Help define “hallucination types” and

build reproducible test suites for failure tracking. You might

thrive in this role if you

Are pursuing a

Bachelor’s degree in Computer Science, Engineering, Math, Physics,

or a related field. Have completed coursework in machine learning,

natural language processing, or AI systems. Enjoy debugging model

outputs and understanding why a chatbot gave a weird answer. Have

worked on side projects involving chatbots, retrieval-based

systems, or LLMs. Can write clean Python code and think critically

about accuracy, context, and grounding. Bonus Points

Experience with LangChain, LlamaIndex, or

RAG architectures. Familiarity with evaluation frameworks like

lm-eval-harness, RAGAS, or custom harnesses. Knowledge of vector

databases (e.g., Qdrant) and prompt engineering techniques.

Interest in safety, reliability, or interpretability of AI agents.

This is a 12-month duration COOP / Intern hybrid role (four days

in-office) reporting directly to the Technology Director. The

salary range for this role is $20–$30 per hours. Interested

candidates are invited to submit their cover letter and resume by

selecting the link : Themis Intelligence values a diverse workplace

and strongly encourages women, people of all races, color, creed,

ancestry, ethnic origin, sexual orientation, gender identity or

expression, age, religion, national origin, citizenship status,

disability, marital status, family status, and those with

disabilities to apply. We use AI tools to help streamline parts of

our recruitment process, but every application is reviewed by a

member of our team. Themis is an equal opportunity employer. We are

committed to providing accommodations for persons with

disabilities. If you require accommodation, we will work with you

to meet your needs. While we appreciate the interest of all

applicants, only those selected for an interview will be

contacted.

J-18808-Ljbffr