Business Systems Analysis Analyst

LLM/agentic AI

Role summary

You will build and support agentic AI workflows—LLM-powered agents that can plan multi-step tasks, use tools/APIs, retrieve knowledge, and act with guardrails. You’ll work with senior engineers and SMEs to ship prototypes and production features for internal users and clients.

What you’ll do

• Design, implement, and test agent workflows (planning, memory, tool use, retry/rollback) using frameworks such as LangGraph, LangChain, AutoGen, CrewAI or equivalent.
• Build tooling integrations (search/RAG over vector stores like FAISS/Pinecone, REST/GraphQL APIs, databases, web/email/calendar, ticketing systems).
• Create robust prompting (system/task prompts, tool schemas), plus evaluation harnesses (unit tests, golden sets, LLM-as-judge, offline/online evals).
• Add guardrails for safety, PII handling, and policy compliance; instrument observability & tracing (e.g., LangSmith, OpenTelemetry logs).
• Optimize cost/latency/reliability (caching, batching, function-calling, streaming, fallbacks).
• Package and deploy services (Docker, basic CI/CD; cloud on AWS/Azure/GCP with secrets management).
• Write concise tech docs and demo the work; support pilot rollouts and collect feedback.

Must-have qualifications

• Well trained or ~0 to 6 months hands-on experience with LLMs/agents through projects, internship, client POCs, or formal training.  
• Practical knowledge of at least one of: LangGraph/LangChain/AutoGen/CrewAI, and one of OpenAI/Anthropic/Google/Meta LLMs (function calling/tool use).
• Strong Python (or TypeScript/Node) fundamentals; working with REST APIs, JSON, and simple data pipelines.
• Experience implementing RAG: chunking, embeddings, vector search, relevance evaluation.
• Understanding of prompt engineering, evaluation, and basic guardrails/safety concepts.
• Git proficiency and clear documentation habits.

Nice to have

• Basic frontend for agent UIs (React) or chat surfaces (Teams/Slack).
• Cloud exposure (Azure OpenAI, AWS Bedrock, GCP Vertex), Docker, CI/CD.
• Observability (LangSmith, Phoenix, Weights & Biases) and cost monitoring.
• Data skills: SQL, pandas, lightweight ETL.
• Domain exposure (e.g., finance ops, customer support, procurement) to ground tools and workflows.