Tech-Driven Performance Reviews

2025 is the Year of AI Agents, not just standalone LLMs.   Anthropic has been using this new approach called Multi-Component AI Agents with Feedback Loops.   AI Agents go beyond basic LLMs with structured parts that work together, letting them solve problems on their own and get better with practice.   Here's how AI Agents work: 1️⃣ Perception Layer Agents take in information through special modules that understand context and track what's happening, helping them see the full picture.   2️⃣ Cognitive Core The thinking and planning parts work together, mixing logical reasoning with goal-setting to make smart choices.   3️⃣ Execution Framework A dedicated action layer picks the best moves and uses outside tools, while checking how well things are working.   4️⃣ Learning Loop System Key feedback paths connect what happened to memory storage, creating a cycle that makes the agent better over time.   5️⃣ Multi-Tool Integration Special outside tools like Web, Code, and API access let an agent do more than what's built in.   Whether you're handling complex workflows or tackling multi-step problems, AI Agents deliver better results through their connected design, giving you more reliable performance and flexible responses.   Here's how AI Agents differ from traditional LLMs:   LLMs: Work as single units focused mainly on generating text Process inputs and create outputs without structured decision paths Don't have clear ways to learn from their results   AI Agents: Function as multi-part systems with specialized modules for different thinking tasks Include clear feedback paths linking results back to reasoning Use outside tools through purpose-built connection points   Understanding these distinctions helps when building systems that can handle complex tasks with less human input.   AI Agents aren't just different; they're more advanced systems:   ✅ Process information through purpose-built thinking ✅ Learn constantly from their results ✅ Change strategies based on what worked before   The feedback loop design matters. It turns one-time interactions into ongoing learning relationships, creating systems that actually get better with time.   Over to you: What tasks do you think would benefit the most for AI Agents?

It’s performance review season – including at Glean. I often hear the same thing from leaders: the process is broken. Reviews are biased, overly focused on what’s recent or easy to recall, and require exhausting manual effort, digging through tools to piece together past work. I appreciated David Ferrucci’s recent piece in Fortune, which explores what happens when AI doesn’t just help us work, but actually evaluates that work. In his case, AI made “invisible” effort visible. At Glean, that future is already here. This cycle, every employee is encouraged to use our self-review agents to help draft their reviews. Our engineering self-review agent, for example, automatically pulls contributions from GitHub, Jira, Slack, and Drive to generate a structured, evidence-backed summary. And our customers are doing the same thing. By grounding reviews in actual work – not memory – they’re making the process faster, fairer, and more accurate. With the right AI, performance reviews stop testing your memory, and start reflecting your impact. https://lnkd.in/gjV3cnrZ

LLM applications are frustratingly difficult to test due to their probabilistic nature. However, testing is crucial for customer-facing applications to ensure the reliability of generated answers. So, how does one effectively test an LLM app? Enter Confident AI's DeepEval: a comprehensive open-source LLM evaluation framework with excellent developer experience. Key features of DeepEval: - Ease of use: Very similar to writing unit tests with pytest. - Comprehensive suite of metrics: 14+ research-backed metrics for relevancy, hallucination, etc., including label-less standard metrics, which can quantify your bot's performance even without labeled ground truth! All you need is input and output from the bot. See the list of metrics and required data in the image below! - Custom Metrics: Tailor your evaluation process by defining your custom metrics as your business requires. - Synthetic data generator: Create an evaluation dataset synthetically to bootstrap your tests My recommendations for LLM evaluation: - Use OpenAI GPT4 as the metric model as much as possible. - Test Dataset Generation: Use the DeepEval Synthesizer to generate a comprehensive set of realistic questions! Bulk Evaluation: If you are running multiple metrics on multiple questions, generate the responses once, store them in a pandas data frame, and calculate all the metrics in bulk with parallelization. - Quantify hallucination: I love the faithfulness metric, which indicates how much of the generated output is factually consistent with the context provided by the retriever in RAG! CI/CD: Run these tests automatically in your CI/CD pipeline to ensure every code change and prompt change doesn't break anything. - Guardrails: Some high-speed tests can be run on every API call in a post-processor before responding to the user. Leave the slower tests for CI/CD. 🌟 DeepEval GitHub: https://lnkd.in/g9VzqPqZ 🔗 DeepEval Bulk evaluation: https://lnkd.in/g8DQ9JAh Let me know in the comments if you have other ways to test LLM output systematically! Follow me for more tips on building successful ML and LLM products! Medium: https://lnkd.in/g2jAJn5 X: https://lnkd.in/g_JbKEkM #generativeai #llm #nlp #artificialintelligence #mlops #llmops

Explaining the Evaluation method LLM-as-a-Judge (LLMaaJ). Token-based metrics like BLEU or ROUGE are still useful for structured tasks like translation or summarization. But for open-ended answers, RAG copilots, or complex enterprise prompts, they often miss the bigger picture. That’s where LLMaaJ changes the game. 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝘁? You use a powerful LLM as an evaluator, not a generator. It’s given: - The original question - The generated answer - And the retrieved context or gold answer 𝗧𝗵𝗲𝗻 𝗶𝘁 𝗮𝘀𝘀𝗲𝘀𝘀𝗲𝘀: ✅ Faithfulness to the source ✅ Factual accuracy ✅ Semantic alignment—even if phrased differently 𝗪𝗵𝘆 𝘁𝗵𝗶𝘀 𝗺𝗮𝘁𝘁𝗲𝗿𝘀: LLMaaJ captures what traditional metrics can’t. It understands paraphrasing. It flags hallucinations. It mirrors human judgment, which is critical when deploying GenAI systems in the enterprise. 𝗖𝗼𝗺𝗺𝗼𝗻 𝗟𝗟𝗠𝗮𝗮𝗝-𝗯𝗮𝘀𝗲𝗱 𝗺𝗲𝘁𝗿𝗶𝗰𝘀: - Answer correctness - Answer faithfulness - Coherence, tone, and even reasoning quality 📌 If you’re building enterprise-grade copilots or RAG workflows, LLMaaJ is how you scale QA beyond manual reviews. To put LLMaaJ into practice, check out EvalAssist; a new tool from IBM Research. It offers a web-based UI to streamline LLM evaluations: - Refine your criteria iteratively using Unitxt - Generate structured evaluations - Export as Jupyter notebooks to scale effortlessly A powerful way to bring LLM-as-a-Judge into your QA stack. - Get Started guide: https://lnkd.in/g4QP3-Ue - Demo Site: https://lnkd.in/gUSrV65s - Github Repo: https://lnkd.in/gPVEQRtv - Whitepapers: https://lnkd.in/gnHi6SeW

Meta and JPMorgan Chase announced they’d be using AI for annual performance reviews. As if the process couldn’t get any more performative and detached from reality. Now we’re adding AI slop to employee feedback. The goal is to summarize a year’s worth of work with the help of AI, but the challenge is that most AI doesn’t have the domain expertise required to write a high-quality performance review. Just because it can generate something in the format of an employee review or summarize notes in an employee review style doesn’t mean it has the knowledge to do it well. AI lacks the context to evaluate the intent and define the outcome. That’s why AI alone is rarely enough to support enterprise or customer use cases. AI doesn’t have the evaluation criteria for what makes a performance review high-quality. It doesn’t have access to comprehensive long and short-term employee outcomes to know what about a review improves them. It lacks context about how employees respond to the feedback it generates. AI doesn’t understand the intent of the manager writing the review or their management style. There are two levels of information required to support this use case: General Information: Domain knowledge about what makes performance reviews effective. Personalized Information: Domain knowledge about the employee and manager that personalizes the review to fit the unique nuances of both people involved. AI alone isn’t enough to support the performance review use case, so it’s only one part of the agentic workflow. The decision and workflow chains must be mapped. Resources must be provided at each link in the chain. Intent and outcomes must be fully defined upfront. Agents require new design paradigms that go beyond AI, or the result is slop.

The AI Assessment Effect Candidates often tend to adjust their answers or behavior to match what they believe the “ideal candidate” profile looks like. A new study published earlier this month found that when candidates believe they’re being assessed by artificial intelligence, they emphasize analytical skills and downplay their intuitive and emotional skills. This so-called “AI assessment effect” stems from the widespread assumption that AI-based evaluations prioritize rational, data-driven attributes over human-centric abilities. Researchers warn that if job seekers tailor their behavior to what they think AI values, their true competencies and personalities may remain hidden, undermining the integrity of the recruitment process. In addition if most candidates assume AI favors analytical traits, the talent pipeline could become increasingly uniform, limiting diversity and reducing the variety of perspectives within organizations. The researchers recommend 1) Radical transparency: Don’t just disclose that AI is used in assessments—be explicit about what it evaluates. Clearly communicate that your AI values a range of traits, including creativity, emotional intelligence, and intuitive problem-solving. Share examples of successful candidates who excelled by showcasing these qualities. 2) Regular behavioral audits: Go beyond demographic bias checks. Look for patterns of behavioral adaptation: Are candidates’ responses becoming more homogeneous over time? Is there a noticeable shift toward analytical self-presentation at the expense of other valuable traits? 3) Hybrid assessment models: Combine AI and human judgment to ensure a more balanced and holistic evaluation of candidates. See research published in the June issue of the Proceedings of the National Academy of Arts and Sciences. https://lnkd.in/ebtD4HBd

She was one of our brightest talents Smart. Committed. A quiet force that lifted the whole team And then... she resigned No warning. No second thoughts. Just… gone. We were stunned. She had everything: a promising future, fair pay, great feedback. So we asked her why. Her words hit like a punch: "I didn’t feel seen. I didn’t feel like we mattered." That moment changed everything. Because the truth is, we missed the signs: - Her engagement score had dropped - Her internal applications went nowhere - She kept going the extra mile with no recognition We had the data. We just didn’t use it wisely. Today, we have no excuse. AI and predictive analytics give us a head start. They help us spot patterns before they become problems: - Who might be silently disengaging? - Where are we overlooking skills and potential? - Are we creating an inclusive space where everyone feels they belong? This isn’t about replacing human connection, it’s about deepening it. When we pair data with empathy, we lead smarter, faster, and more human. Because great HR doesn’t just prevent risks. It unlocks possibility. If we reinforce our data and tools, we can spend even more time on what matters most: making sure people remain at the heart of our organizations. #Talents #PredictiveHR #DataDrivenLeadership #EmployeeExperience #humanresources

User Feedback Loops: the missing piece in AI success? AI is only as good as the data it learns from -- but what happens after deployment? Many businesses focus on building AI products but miss a critical step: ensuring their outputs continue to improve with real-world use. Without a structured feedback loop, AI risks stagnating, delivering outdated insights, or losing relevance quickly. Instead of treating AI as a one-and-done solution, companies need workflows that continuously refine and adapt based on actual usage. That means capturing how users interact with AI outputs, where it succeeds, and where it fails. At Human Managed, we’ve embedded real-time feedback loops into our products, allowing customers to rate and review AI-generated intelligence. Users can flag insights as: 🔘Irrelevant 🔘Inaccurate 🔘Not Useful 🔘Others Every input is fed back into our system to fine-tune recommendations, improve accuracy, and enhance relevance over time. This is more than a quality check -- it’s a competitive advantage. - for CEOs & Product Leaders: AI-powered services that evolve with user behavior create stickier, high-retention experiences. - for Data Leaders: Dynamic feedback loops ensure AI systems stay aligned with shifting business realities. - for Cybersecurity & Compliance Teams: User validation enhances AI-driven threat detection, reducing false positives and improving response accuracy. An AI model that never learns from its users is already outdated. The best AI isn’t just trained -- it continuously evolves.

That’s the thing about feedback—you can’t just ask for it once and call it a day. I learned this the hard way. Early on, I’d send out surveys after product launches, thinking I was doing enough. But here’s what happened: responses trickled in, and the insights felt either outdated or too general by the time we acted on them. It hit me: feedback isn’t a one-time event—it’s an ongoing process, and that’s where feedback loops come into play. A feedback loop is a system where you consistently collect, analyze, and act on customer insights. It’s not just about gathering input but creating an ongoing dialogue that shapes your product, service, or messaging architecture in real-time. When done right, feedback loops build emotional resonance with your audience. They show customers you’re not just listening—you’re evolving based on what they need. How can you build effective feedback loops? → Embed feedback opportunities into the customer journey: Don’t wait until the end of a cycle to ask for input. Include feedback points within key moments—like after onboarding, post-purchase, or following customer support interactions. These micro-moments keep the loop alive and relevant. → Leverage multiple channels for input: People share feedback differently. Use a mix of surveys, live chat, community polls, and social media listening to capture diverse perspectives. This enriches your feedback loop with varied insights. → Automate small, actionable nudges: Implement automated follow-ups asking users to rate their experience or suggest improvements. This not only gathers real-time data but also fosters a culture of continuous improvement. But here’s the challenge—feedback loops can easily become overwhelming. When you’re swimming in data, it’s tough to decide what to act on, and there’s always the risk of analysis paralysis. Here’s how you manage it: → Define the building blocks of useful feedback: Prioritize feedback that aligns with your brand’s goals or messaging architecture. Not every suggestion needs action—focus on trends that impact customer experience or growth. → Close the loop publicly: When customers see their input being acted upon, they feel heard. Announce product improvements or service changes driven by customer feedback. It builds trust and strengthens emotional resonance. → Involve your team in the loop: Feedback isn’t just for customer support or marketing—it’s a company-wide asset. Use feedback loops to align cross-functional teams, ensuring insights flow seamlessly between product, marketing, and operations. When feedback becomes a living system, it shifts from being a reactive task to a proactive strategy. It’s not just about gathering opinions—it’s about creating a continuous conversation that shapes your brand in real-time. And as we’ve learned, that’s where real value lies—building something dynamic, adaptive, and truly connected to your audience. #storytelling #marketing #customermarketing

LLMs are great at many things; however, continuous decision-making, which is needed for agentic work, is not one of them! A team of researchers has developed SAGE (Self-evolving Agents with Reflective and Memory-augmented Abilities), an innovative framework to enhance large language models' decision-making capabilities in complex, dynamic environments. The backbone of SAGE consists of three main components: - Iterative Feedback Mechanism - Reflection Module - Memory Management System Iterative Feedback Mechanism The Iterative Feedback Mechanism involves three key agents: - User (U): Initiates tasks and provides initial input. - Assistant (A): Generates text and actions based on environmental observations. - Checker (C): Evaluates the assistant's output and provides feedback. The iterative process continues until the checker deems the assistant's output correct or the iteration limit is reached. This mechanism allows for continuous improvement of the assistant's responses. Reflection Module The Reflection Module enables the assistant to analyze past experiences and store learned lessons in memory. It provides a sparse reward signal, such as binary success states, and generates self-reflections. These reflections are more informative than scalar rewards and are stored in the agent's memory for future reference. Memory Management System SAGE employs a sophisticated memory management system divided into two types: - Short-Term Memory (STM): Stores immediately relevant information for the current task. It's highly volatile and frequently updated. - Long-Term Memory (LTM): Retains information deemed important for future tasks. It has a larger capacity and can store information for extended periods. A key innovation in SAGE is also the MemorySyntax method, which combines the Ebbinghaus forgetting curve with linguistic knowledge. This approach optimizes the agent's memory and external storage management by: - Adjusting sentence structure based on part-of-speech priority. - Simulating human memory and forgetting mechanisms. - Managing the transfer of information between working memory (Ms) and long-term memory (Ml).