But most of them have one major problem:

They forget everything.

You ask the agent something… then ask a follow-up question… and suddenly it behaves like it has amnesia.

That’s because many developers focus only on:

• prompts,

• tools,

• workflows,

• and LLM selection…

…but completely ignore memory systems.

The reality is:

Memory is what transforms an LLM into an actual AI agent.

Without memory:

• agents feel dumb,

• conversations break,

• personalization disappears,

• and long-running tasks become impossible.

Before implementing any AI agent, you must understand the different memory types, when to use them, and what should (and should not) be stored.

In this article, we’ll break down:

• Short-term memory

• Long-term memory

• Semantic memory

• Episodic memory

• Working memory

• Retrieval memory And how modern AI systems like OpenAI and Anthropic likely structure memory internally.

Why Memory Matters in AI Agents

Imagine hiring a human assistant who:

• forgets your name every minute,

• never remembers previous tasks,

• and resets after every conversation.

That assistant would be useless.

Yet that’s exactly how many AI agents behave today.

Memory enables agents to:

• maintain conversation context,

• remember user preferences,

• learn from previous interactions,

• improve future responses,

• and perform long-running autonomous tasks.

This is the difference between:

• a chatbot,

• and a real AI assistant.

1. Short-Term Memory (Conversation Memory)

What It Is

Short-term memory stores:

• recent messages,

• current task context,

• active reasoning steps,

• and temporary conversation state.

This memory usually lives inside:

• the context window,

• Redis,

• in-memory cache,

• or session storage.

Example

User says:

“Help me write a LinkedIn post about AI agents.”

Then later:

“Make it shorter.”

The AI needs short-term memory to understand: “it” = the LinkedIn post.

Without it, the agent gets confused.

Common Mistake

Many developers dump entire conversations into prompts.

This:

• increases token cost,

• slows responses,

• and eventually exceeds context limits.

Good agents summarize and compress short-term memory over time.

2. Long-Term Memory (Persistent Memory)

What It Is

Long-term memory stores information across sessions.

This includes:

• user preferences,

• goals,

• writing style,

• past projects,

• recurring workflows,

• and important facts.

Usually stored in:

• vector databases,

• PostgreSQL,

• graph databases,

• or knowledge stores.

Example

If a user always writes:

• backend engineering blogs,

• AI tutorials,

• and YouTube scripts…

…the agent can remember this and personalize future outputs automatically.

That’s how assistants start feeling “smart.”

3. Semantic Memory

What It Is

Semantic memory stores facts and knowledge.

Think of it like:

• concepts,

• relationships,

• expertise,

• and learned information.

Example

The agent remembers:

• LangChain is an AI framework,

• PostgreSQL is a database,

• Redis is used for caching.

This is knowledge-based memory.

Humans use semantic memory too.

4. Episodic Memory

What It Is

Episodic memory stores experiences and past interactions.

Instead of remembering facts, the agent remembers events.

Example

The agent remembers:

“Last week the user struggled with Docker networking.”

That historical experience helps future responses.

This is one of the most important memory types for personalization.

5. Working Memory

What It Is

Working memory is temporary reasoning memory.

It exists only while solving a task.

The agent may store:

• intermediate reasoning,

• calculations,

• plans,

• or execution steps.

Once the task finishes, this memory may disappear.

Example

While generating SQL:

1. Understand schema

2. Build query

3. Validate query

4. Execute safely

The intermediate steps live in working memory.

6. Retrieval Memory (RAG Memory)

What It Is

Instead of storing everything directly in prompts, the agent retrieves relevant information when needed.

This powers:

• RAG systems,

• document agents,

• and enterprise AI assistants.

Example

User asks:

“Summarize my uploaded PDF.”

The agent:

1. retrieves relevant chunks,

2. injects them into context,

3. then answers.

This avoids context overload.

The Real Challenge: What Should Be Stored?

This is where most AI agent systems fail.

Not everything deserves long-term memory.

Good memory systems decide:

• what to remember,

• what to forget,

• and what to summarize.

Good Things to Store

• User preferences

• Long-term goals

• Writing style

• Repeated workflows

• Important project details

Bad Things to Store

• Temporary emotions

• One-time requests

• Sensitive information

• Random conversational noise

A Practical AI Agent Memory Architecture

A production-grade AI agent often looks like this:

This layered architecture is what makes modern AI assistants scalable.

How Modern AI Systems Likely Handle Memory

Companies like OpenAI and Anthropic likely use:

• session memory,

• persistent user memory,

• retrieval systems,

• summarization pipelines,

• and memory ranking systems.

The hard part is not storing memory.

The hard part is:

• deciding importance,

• retrieval timing,

• compression,

• and relevance filtering.

Memory engineering is becoming its own field.

Final Thoughts

Most developers think AI agents are about:

• better prompts,

• bigger models,

• or more tools.

But memory is what actually creates continuity, intelligence, and personalization.

The future of AI agents won’t belong to the models with the largest context windows.

It will belong to agents that:

• remember intelligently,

• forget strategically,

• and learn continuously.

If you truly want to build production-grade AI agents…

start with memory architecture first.

That’s why cron jobs matter more than most developers realize. They are the invisible safety net keeping systems alive in production.

Below are 5 cron jobs every backend server should have.

#1. Automated Backups

Regularly backing up your data is non-negotiable. This is arguably one of the most critical uses of cron jobs in any backend system.

In production, things rarely break on purpose—they break due to small, unexpected mistakes. A deleted record, a faulty deployment, or even a storage corruption issue can wipe out critical data in seconds.

A backup cron job ensures that no matter what happens, there is always a recent version of your data that can be restored. It quietly runs in the background, taking snapshots of your database and storing them safely in external storage.

#2. Log Cleanup & Rotation

Logs are extremely useful until they aren’t. Over time, they quietly grow into massive files that start consuming disk space without any warning.

A log cleanup cron job ensures that old logs are either deleted or compressed regularly. Without this, servers can suddenly run out of storage and crash unexpectedly.

This job is less about performance optimization and more about survival. It keeps your system stable by preventing silent resource exhaustion.

#3. Server Health Checks

Your server rarely tells you when it’s about to fail—it just does. That’s why continuous health checks are essential.

This cron job periodically checks CPU usage, memory consumption, disk space, and service availability. If anything crosses safe limits, it triggers alerts before users are affected.

It acts like a silent observer, constantly watching the system so you don’t have to manually inspect it every few hours.

#4. Cache Cleanup

Every backend system generates temporary files—uploads, cached responses, processing artifacts, and more. Most of these are never cleaned automatically.

Over time, these files accumulate and slowly degrade system performance or fill up storage completely.

A cleanup cron job ensures that anything temporary truly stays temporary. It removes stale files and keeps the system lean, fast, and predictable.

#5. Retry Failed Jobs

Not everything works on the first attempt. API calls fail, webhooks timeout, and background jobs occasionally break due to external dependencies.

Instead of losing that work permanently, a retry cron job reprocesses failed tasks at regular intervals.

This is what makes modern backend systems resilient. It ensures that temporary failures don’t turn into permanent data loss or broken workflows.

Final Thoughts

Cron jobs are often overlooked because they don’t feel “core” to the product. But in reality, they are what make production systems stable, reliable, and self-healing.

Most backend failures don’t come from code—they come from missing automation.

And these 5 cron jobs quietly prevent that from happening.

What if you could query your database like this:

"Show me top 10 users by revenue"

…and get instant results—without writing SQL?

Welcome to the world of AI-powered database agents.

In this tutorial, you'll learn how to build a secure PostgreSQL AI agent using:

• 🧩 LangChain — for agent orchestration and tool chaining

• 🦙 Ollama — to run a local LLM with zero API cost

• 🐘 PostgreSQL — as the target database

• 🛡️ Custom SQL safety guard — to block destructive queries

By the end, you'll have a production-ready AI database assistant that understands natural language and safely executes SQL queries.

💻 Source Code: https://github.com/icon-gaurav/postgres-agent

🤖 What is a PostgreSQL AI Agent?

A PostgreSQL AI Agent is an LLM-powered system that:

• Converts natural language → SQL queries

• Executes queries on PostgreSQL

• Returns structured results

👉 Think of it as ChatGPT for your database, but controlled and safe.

⚙️ Tech Stack

🧱 Architecture

🔌 Step 1: PostgreSQL Connection

First, configure your database connection using psycopg2:

import psycopg2
 
DB_CONFIG = {
    "host": "localhost",
    "port": 5432,
    "database": "postgres",
    "user": "postgres",
    "password": "root",
}
 
def get_connection():
    return psycopg2.connect(**DB_CONFIG)

🔐 Security tip: In production, load credentials from environment variables or a secrets manager — never hardcode passwords.

🛠️ Step 2: Create Database Tools

LangChain agents interact with the world through tools — Python functions decorated with @tool that the LLM can invoke by name.

📋 List Tables Tool

@tool
def list_tables() -> str:
    """List all tables in the database."""
    conn = get_connection()
    try:
        cur = conn.cursor()
        cur.execute("""
            SELECT table_name FROM information_schema.tables
            WHERE table_schema = 'public'
        """)
        tables = [row[0] for row in cur.fetchall()]
        return f"Tables: {', '.join(tables)}" if tables else "No tables found."
    finally:
        conn.close()

This enables dynamic schema discovery — the agent doesn't need hardcoded table names.

📑 Get Table Schema

@tool
@tool
def get_table_schema(table_name: str) -> str:
    """Get the schema (columns and types) of a specific table."""
    conn = get_connection()
    try:
        cur = conn.cursor()
        cur.execute("""
            SELECT column_name, data_type, is_nullable
            FROM information_schema.columns
            WHERE table_schema = 'public' AND table_name = %s
            ORDER BY ordinal_position
        """, (table_name,))
        columns = cur.fetchall()
        if not columns:
            return f"Table '{table_name}' not found."
        schema = "\n".join([f"  {col[0]} ({col[1]}, nullable={col[2]})" for col in columns])
        return f"Schema for '{table_name}':\n{schema}"
    finally:
        conn.close()

Allows the agent to understand:

• Column names

• Data types

• Constraints

⚡ Execute SQL Tool

@tool
def execute_sql(query: str) -> str:
    """Execute a SQL query against the PostgreSQL database and return results. Use this for SELECT queries."""
    is_safe, reason = validate_read_only_sql(query)
    if not is_safe:
        return f"Safety Guard: Blocked query. {reason}"

    conn = get_connection()
    try:
        cur = conn.cursor()
        cur.execute(query)
        if cur.description:
            columns = [desc[0] for desc in cur.description]
            rows = cur.fetchall()
            if not rows:
                return "Query returned no results."
            result = " | ".join(columns) + "\n"
            result += "\n".join([" | ".join(str(v) for v in row) for row in rows[:50]])
            if len(rows) > 50:
                result += f"\n... ({len(rows)} total rows)"
            return result
        else:
            conn.commit()
            return f"Query executed successfully. Rows affected: {cur.rowcount}"
    except Exception as e:
        conn.rollback()
        return f"SQL Error: {e}"
    finally:
        conn.close()

This is the core execution layer.

Step 3: SQL Safety Guard — Prevent Destructive Queries

Allowing an LLM to run arbitrary SQL is a critical security risk. The SQL safety guard validates every query before execution.

Read-Only Allowlist

Only these SQL statement types are permitted:

🧼 Normalize Queries

We remove:

• Comments

• Strings

• Hidden injections

👉 This ensures safe AI execution in production environments.

🧠 Step 4: Setup Ollama (Local LLM)

Ollama lets you download and run large language models entirely on your own machine — no cloud account, no API key, no usage fees.

📚 Official Resources:

• 🌐 Website: ollama.com

• 📖 Documentation: docs.ollama.com

• 🗂️ Model Library: ollama.com/library

• 🐙 GitHub: github.com/ollama/ollama

🔽 Pulling a Model

Once Ollama is running, pull the model used in this project:

ollama pull qwen2.5:7b

You can verify it's available with:

ollama list

Browse all available models at ollama.com/library. Some good alternatives for SQL agents:

📦 Installing the LangChain Ollama Package

The LangChain integration for Ollama lives in the dedicated langchain-ollama package:

pip install langchain-ollama

📖 Package References:

• 📦 PyPI: pypi.org/project/langchain-ollama

• 🔗 LangChain Docs: docs.langchain.com — ChatOllama

• 📐 API Reference: reference.langchain.com/python/langchain-ollama

⚙️ Configuring ChatOllama

Now initialize the LLM in your Python code:

from langchain_ollama import ChatOllama
 
llm = ChatOllama(model="qwen2.5:7b", temperature=0)

Setting temperature=0 makes the model deterministic — essential for reliable SQL generation. You can tune other key parameters as needed:

Why Ollama?

• ✅ No API cost

• ✅ Runs fully locally — data never leaves your machine

• ✅ Privacy-friendly — ideal for sensitive database workloads

• ✅ Fast inference with GPU support

• ✅ Supports dozens of open-source models

🔗 Step 5: Create LangChain Agent

tools = [list_tables, get_table_schema, execute_sql]
agent = create_agent(llm, tools)

LangChain allows the AI to:

• Decide which tool to use

• Chain multiple steps

• Reason dynamically

💬 Step 6: Interactive Chat Loop

while True:
    user_input = input("\nYou: ").strip()
    if user_input.lower() in ("exit", "quit"):
        print("Goodbye!")
        break
    if not user_input:
        continue

This makes your agent:

• Conversational

• Stateful

• Easy to debug

🧾Step 7: Debugging & Observability

Visibility into what the agent is doing is essential for development. This helper function prints each tool call and its result:

def print_turn_details(messages: list[BaseMessage]) -> None:
    final_response = ""

    for message in messages:
        if isinstance(message, AIMessage):
            for tool_call in message.tool_calls:
                tool_name = tool_call.get("name", "unknown_tool")
                tool_args = format_tool_payload(tool_call.get("args", {}))
                print(f"\nTool call: {tool_name}({tool_args})")

            content = format_content(message.content).strip()
            if content:
                final_response = content

        elif isinstance(message, ToolMessage):
            tool_name = getattr(message, "name", None) or "tool"
            tool_output = format_content(message.content).strip() or "(no output)"
            print(f"\nTool response [{tool_name}]: {tool_output}")

    if final_response:
        print(f"\nAgent: {final_response}")
    else:
        print("\nAgent: I couldn't generate a response.")

Shows:

• Tool calls

• Tool outputs

• Final response

👉 This is extremely useful for debugging agent behavior.

🧪 Example Queries

Try asking:

• "List all tables"

• "Show schema of users table"

• "Get top 5 users by revenue"

• "How many orders were placed last month?"

Real-World Use Cases for a Natural Language Database Agent

This architecture can power a wide range of applications:

📊 AI-Powered Analytics Dashboards — let non-technical stakeholders query live data in plain English without learning SQL.

💬 Internal Data Chatbots — embed in Slack or Microsoft Teams so product and ops teams can self-serve data questions.

🧾 Automated Reporting — schedule the agent to answer recurring questions and generate daily or weekly reports.

🏢 SaaS Admin Panels — give your ops team a natural language interface to your product database.

🤖 AI Copilots for Data Teams — speed up analyst workflows by auto-generating SQL drafts from plain-English specs.

🎯 Conclusion

You've built more than just a demo. This is a secure, extensible AI database agent that can be used in real-world applications.

Key Takeaways:

• LangChain simplifies agent workflows

• Ollama enables local LLM execution

• SQL safety is critical

• Tool-based architecture = powerful AI agents

The core insight: wrapping your database in typed, well-described LangChain tools gives the LLM exactly the context it needs to generate correct SQL — without ever exposing raw database access.

📦 Full Source Code — complete working implementation from this tutorial

The Problem With Building Alone

Before I talk about OpenClaw, let me paint the picture.

When you're building a SaaS product by yourself, you're not just a developer. You're the researcher, the architect, the QA engineer, the SEO specialist, and — if you're lucky — the person who actually writes code. Every decision defaults to you. Every skill gap becomes a blocker.

Want to add authentication? Better research OAuth providers, understand security best practices, and implement it right. Need to figure out email deliverability? Time to go down a 3-hour rabbit hole. Building alone means you're constantly context-switching between "figuring things out" and "actually building."

I know because I've been there. Building ShiftMailer — my product for AI-powered email marketing — meant I had to wear all these hats. And honestly? It was exhausting.

Then Came OpenClaw

OpenClaw isn't just another AI chatbot. It's an AI agent framework that can actually do things — read files, run commands, search the web, analyze code, and coordinate with other tools. It's like having a team member who doesn't sleep, doesn't complain, and can spin up new skills on demand.

Here's what I learned after using it for ShiftMailer development.

What Actually Works

1. Research That Goes Beyond Google

I used to spend hours researching trends, comparing tools, and validating ideas. Now I can ask OpenClaw to:

• Find current trends in email marketing automation

• Compare pricing models of competitors

• Research technical decisions (like multi-tenancy approaches)

It doesn't just give me links — it synthesizes information and gives me actionable insights. This alone saved me days of scattered research.

2. Code That I Still Own

Here's something important: I know how to code. I'm a backend engineer. But that doesn't mean I want to write every boilerplate or debug every edge case alone.

OpenClaw helps me:

• Write faster — It handles the repetitive stuff so I focus on the interesting parts

• Review my code — Fresh eyes catch bugs I missed

• Explore new patterns — I can ask "how would you approach this with Node.js streams?" and get working examples

It's not replacing my skills. It's amplifying them.

3. Real-World Analysis

One surprise: OpenClaw analyzed my website's SEO and gave me specific suggestions. Not generic advice — actual, implementable recommendations based on my content. That's not something I expected an AI agent to do well, but it handled it.

The Honest Take: What Doesn't Work

I promised you an honest review, so here it is:

Orchestration is Hard

OpenClaw is great at analyzing requirements and using individual tools well. But when it comes to coordinating complex multi-step workflows? Sometimes it struggles. Things that would be trivial for a human — "okay, first do A, then B, but only if C worked" — can get messy.

This means I'm still the conductor. The agent executes well, but I'm the one keeping the orchestra in sync.

Security: You Gotta Be Careful

Here's the thing: OpenClaw has access to my files, my environment, my code. That's powerful, but it's also a responsibility.

For now, I'm careful about:

• Not giving agents unrestricted external access

• Reviewing code before shipping

• Keeping sensitive configs isolated

This isn't a criticism of OpenClaw — it's just smart practice. When you're delegating to an AI, trust but verify.

The Comparison: Traditional Cofounder vs. AI Agent

People often ask: "Isn't this like having a cofounder?"

Not really. Here's the difference:

With a traditional cofounder, I'd need to research every aspect myself, find different people for different skills, and coordinate schedules. With OpenClaw, I can say "I need research on X" and get it done — then switch to "help me debug this API" without friction.

What This Means for Solo Builders

If you're building alone, here's what I'd tell you:

AI agents aren't magic. They're tools. And like any tool, they have limits. But if you learn to work with them — to prompt well, to review outputs, to stay in the loop — you can do way more than you could alone.

ShiftMailer exists today because I stopped trying to do everything myself. I found a way to leverage AI that works with my skills, not instead of them.

What's Next

I'm still figuring out the orchestration piece. I'm still being careful about security. But the gap between "idea" and "working product" has shrunk dramatically.

If you're a solo builder on the fence about AI agents — try it. Start small. See what works for your workflow. You might be surprised what you can ship when you're not alone in the room.

Have questions about building with AI agents or want to share your story? Let's connect — find me on X or check out gauravbytes.hashnode.dev.

That approach works—until you need structure, reliability, or production-grade workflows.

In this post, we’ll explore sequential AI agents, how they differ from normal AI agents, and how you can build a multi-step AI workflow using the Vercel AI SDK.

What Is an AI Agent?

An AI agent is a system that:

1. Takes an input (user query, event, or data)

2. Uses an LLM to reason or generate output

3. Optionally calls tools, APIs, or functions

4. Returns a result or performs an action

In many applications, this entire process happens in one step.

Example:

User asks: “Write a product update email”
→ LLM generates the email in a single response

This works well for simple tasks—but it starts breaking down as complexity grows.

Normal AI Agent (Single-Step Agent)

A normal AI agent typically follows this flow:

Input → LLM → Output

Characteristics

• Single prompt

• Single LLM call

• Minimal or no intermediate state

• Fast and cheap

Example Use Cases

• Chatbots

• Text rewriting

• Summarization

• Simple content generation

Limitations

• Hard to enforce structure

• No explicit reasoning steps

• Poor control over multi-stage workflows

• Difficult to debug or extend

When tasks require planning, validation, transformation, or multiple roles, a single-step agent becomes fragile.

What Is a Sequential AI Agent?

A sequential AI agent breaks a task into multiple ordered steps, where:

• Each step has a clear responsibility

• Output of one step becomes input for the next

• Context accumulates across steps

Input
  ↓
Step 1 (Planner Agent)
  ↓
Step 2 (Executor Agent)
  ↓
Step 3 (Refiner / Validator Agent)
  ↓
Final Output

Instead of asking the model to do everything at once, we guide it through a pipeline.

Normal Agent vs Sequential Agent

Sequential agents trade simplicity for control and reliability.

When Are Sequential AI Agents Beneficial?

Sequential agents are ideal when:

1. Tasks Have Clear Phases

Example:

• Planning

• Writing

• Reviewing

• Formatting

2. Output Must Follow Strict Structure

• Emails

• Reports

• JSON schemas

• Code generation

3. Different “Roles” Are Needed

• Product marketer

• Engineer

• Editor

4. You Want Deterministic Pipelines

• SaaS features

• Automations

• Multi-tenant systems

This is why sequential agents work extremely well for:

• Product update emails

• CRM workflows

• Content pipelines

• Data extraction and transformation

Designing a Sequential Agent (Conceptually)

Let’s say we want to build a content generation agent.

Step 1: Planner Agent

Responsibility:

• Analyze the input

• Break it into structured sections

Output:

{
  "sections": ["Introduction", "Key Points", "Conclusion"]
}

Step 2: Writer Agent

Responsibility:

• Generate content for each section

Input:

• Original user input

• Planner output

Step 3: Refiner Agent

Responsibility:

• Improve tone

• Fix grammar

• Enforce constraints

Each step is predictable and replaceable.

Implementing a Sequential AI Agent with Vercel AI SDK

Step 1: Create the Planner Agent

import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

export async function plannerAgent(input: string) {
  const result = await generateText({
    model: openai("gpt-4.1"),
    prompt: `Analyze the input and create a structured plan.\n\nInput: ${input}`,
  });

  return result.text;
}

Step 2: Create the Writer Agent

export async function writerAgent(plan: string, input: string) {
  const result = await generateText({
    model: openai("gpt-4.1"),
    prompt: `Using the following plan, write detailed content.\n\nPlan:\n${plan}\n\nInput:\n${input}`,
  });

  return result.text;
}

Step 3: Create the Refiner Agent

export async function refinerAgent(content: string) {
  const result = await generateText({
    model: openai("gpt-4.1"),
    prompt: `Refine the following content for clarity and tone.\n\n${content}`,
  });

  return result.text;
}

Step 4: Orchestrate the Sequential Flow

export async function sequentialAgent(input: string) {
  const plan = await plannerAgent(input);
  const draft = await writerAgent(plan, input);
  const finalOutput = await refinerAgent(draft);

  return finalOutput;
}

This orchestration is the heart of a sequential agent.

Benefits of This Approach

• Clear separation of responsibilities

• Easier debugging (inspect each step)

• Reusable agents

• Better output consistency

• Safer production usage

This is especially useful when building AI-powered SaaS features, not demos.

Final Thoughts

Sequential AI agents represent a shift from “ask the model to do everything” to “designing AI workflows.”

With the Vercel AI SDK, building these workflows feels natural and maintainable.

If you’re building:

• AI-first products

• Content pipelines

• Internal tooling

…sequential agents will give you control, clarity, and confidence.

If you’re interested, the next step could be:

• Adding validation agents

• Parallel agents

• Streaming intermediate steps

• Persisting agent state

That’s where AI engineering starts to feel like real software engineering 🚀

• Instead of browsing 20 product pages, users simply ask:
  “Show me red shoes under ₹1500.”

• Instead of navigating a 5-step checkout, they say:
  “Add the second one to my cart and checkout.”

To explore this future, I built a fully functional AI E-Commerce Chatbot using Vercel AI SDK + Gemini, equipped with tools like:

• Fetch Catalog

• Add to Cart

• Checkout Cart

And a UI that responds with card-style product previews, add-to-cart confirmation messages, and interactive checkout prompts.

This blog is a complete to-do guide to help developers build the same chatbot from scratch.

Let’s start building it

🧩 Define Tools for Gemini

Gemini supports function calling, so we define our tools. We will create functions that accept some parameters and return a result based on them. Here, we are using an API call to achieve this, but you can implement any business logic in these functions.

Catalog tool

export async function fetchCatalog({query, maxPrice}: { query: string; maxPrice: number }) {
    const res = await fetch(
        `${process.env.NEXT_PUBLIC_BASE_URL}/api/catalog?query=${query}&maxPrice=${maxPrice}`
    );
    if (!res.ok) throw new Error("Failed to fetch catalog");
    return res.json();
}

Cart tool

export async function addToCart(productId: string, quantity: number) {
    const res = await fetch(`${process.env.NEXT_PUBLIC_BASE_URL}/api/cart`, {
        method: "POST",
        headers: {
            "Content-Type": "application/json",
        },
        body: JSON.stringify({product_id: productId, qty: quantity}),
    });
    if (!res.ok) throw new Error("Failed to add to cart");
    return res.json();
}

Checkout Tool

export async function checkoutCart() {
    const res = await fetch(`${process.env.NEXT_PUBLIC_BASE_URL}/api/checkout`, {
        method: "POST",
    });
    if (!res.ok) throw new Error("Failed to checkout");
    return res.json();
}

🛠️ Create the Tool-Enabled Chat API

This is the heart of the chatbot.

Define the prompt for our AI agent

const SYSTEM_PROMPT = `
You are a shopping assistant AI integrated with tools.

You can help the user browse products, add them to a cart, and checkout.

### Available tools:
1. **fetchCatalog()**
   - Retrieves all products from the catalog.
   - Always call this tool to see what products are available before suggesting items.

2. **addToCart(productId: string)**
   - Adds the specified product to the user's cart.
   - Call this when the user asks to "add", "buy", or "I want this".

3. **checkoutCart()**
   - Processes the order and simulates checkout.
   - Only call this if the user explicitly says "checkout", "place order", or "buy now".

---

### Rules of behavior:
- Never invent products. Only use the results returned by 'fetchCatalog'.
- Always confirm product details (name, price, stock) when suggesting items.
- If the user is vague (e.g., "show me something cool"), fetch the catalog and then suggest a few options.
- Be conversational and friendly, but clearly indicate actions you’re taking.
- After a successful checkout, thank the user and end the flow.

---

### Examples:

**User:** "Show me headphones under $150"  
**Assistant reasoning:** Call 'fetchCatalog', filter results by category + price, then present matching products.  

**User:** "Yes, add the headphones to my cart"  
**Assistant reasoning:** Call 'addToCart(productId)' for that item then show the current cart.  

**User:** "Checkout now"  
**Assistant reasoning:** Call 'checkoutCart' and return the order confirmation.
`

API endpoint to use our chatbot

import {convertToModelMessages, streamText, UIMessage} from "ai";
import {addToCart, checkoutCart, fetchCatalog} from "@/lib/tools";
import {google} from "@ai-sdk/google";
import {NextRequest, NextResponse} from "next/server";
import {z} from 'zod';

export const runtime = "edge";
const gemini = google("models/gemini-2.5-flash-lite");


export async function POST(req: NextRequest) {
    try {
        const {messages}: { messages: UIMessage[] } = await req.json()

        const result = await streamText({
            model: gemini,
            system: SYSTEM_PROMPT,
            messages: convertToModelMessages(messages),
            tools: {
                fetchCatalog: {
                    description: "Retrieves all products from the catalog.",
                    inputSchema: z.object({
                        query: z.string().optional().default(""),
                        maxPrice: z.number().optional().default(1000)
                    }),
                    execute: async ({query, maxPrice}) => {
                        return await fetchCatalog({query, maxPrice});
                    }
                },
                addToCart: {
                    description: "Adds the specified product to the user's cart.",
                    inputSchema: z.object({
                        productId: z.string(),
                        quantity: z.number().optional().default(1)
                    }),
                    execute: async ({productId, quantity}) => {
                        return await addToCart(productId, quantity);
                    }
                },
                checkoutCart: {
                    description: "Process checkout and initiate payment",
                    inputSchema: z.object({}),
                    execute: async () => {
                        return await checkoutCart();
                    }
                }
            }
        });


        return result.toUIMessageStreamResponse({
            onError: errorHandler
        });
    } catch (err) {
        console.error("Error : ", err);
        return NextResponse.json({error: "Internal server error"}, {status: 500});
    }
}

function errorHandler(error: unknown) {
    if (error == null) {
        return 'unknown error';
    }

    if (typeof error === 'string') {
        return error;
    }

    if (error instanceof Error) {
        return error.message;
    }

    return JSON.stringify(error);
}

🎨 Build UI With Card Components

When the LLM returns tool results, your UI displays them as cards so we will implement these components in react

🟥 Product Card

// components/ProductCard.tsx
"use client";

import {useState} from "react";

export default function ProductCard({product, addToCart}: { product: any, addToCart: any }) {
    const [adding, setAdding] = useState(false);
    const [added, setAdded] = useState(false);

    const handleAddToCart = async () => {
        setAdding(true);
        addToCart(product?.id, product?.name)
        setAdded(true)
        setAdding(false);
    };

    return (
        <div className="border rounded-xl p-4 shadow-md flex flex-col items-center gap-2 bg-white">
            <img
                src={product.image}
                alt={product.name}
                className="w-32 h-32 object-cover rounded-lg"
            />
            <h3 className="text-lg font-semibold">{product.name}</h3>
            <p className="text-gray-600">₹{product?.price}</p>
            <button
                onClick={handleAddToCart}
                disabled={adding || added}
                className="bg-blue-600 text-white px-4 py-2 rounded-lg hover:bg-blue-700 disabled:bg-gray-400"
            >
                {adding ? "Adding..." : added ? "Added" : "Add to Cart"}
            </button>
        </div>
    );
}

🟦 Cart Card

"use client";
import {CartItem} from "@/lib/cartStore";
export default function Cart({items = []}: { items?: any }) {
    const total = items?.reduce((sum:number, i:CartItem) => sum + i.price * i.qty, 0);
    return (
        <div className="p-4 bg-white rounded-xl shadow-md w-[320px]">
            <h2 className="text-lg font-bold mb-3">🛒 Your Cart</h2>

            {items.length === 0 && <p className="text-gray-500">Cart is empty</p>}

            <ul className="space-y-3">
                {items.map((item:CartItem) => (
                    <li
                        key={item.productId}
                        className="flex items-center justify-between gap-2 border-b pb-2"
                    >
                        <div className="flex items-center gap-2">
                            {item.image && (
                                <img
                                    src={item.image}
                                    alt={item.name}
                                    className="w-10 h-10 rounded"
                                />
                            )}
                            <div>
                                <p className="font-medium">{item.name}</p>
                                <p className="text-sm text-gray-500">
                                    ₹{item?.price} × {item.qty} =  ${item?.price * item.qty}
                                </p>
                            </div>

                        </div>

                    </li>
                ))}
            </ul>

            {items.length > 0 && (
                <div className="pt-3 flex justify-between font-bold">
                    <span>Total -&nbsp;</span>
                    <span>₹{total ? total.toFixed(2) : "0.00"}</span>
                </div>
            )}
        </div>
    );
}

🟩 Checkout Card

"use client";
import { useState } from "react";
import {OrderItem} from "@/lib/orderStore";

export default function Checkout({order}: { order: OrderItem | null }) {
    const [step, setStep] = useState<"details" | "payment" | "success">("details");
    // mock "processing payment"
    const handlePayment = () => {
        setStep("payment");
        setTimeout(() => setStep("success"), 2000);
    };

    return (
        <div className="p-6 max-w-md mx-auto bg-white rounded-xl shadow-lg">
            <h2 className="text-xl font-bold mb-4">Checkout</h2>

            {step === "details" && (
                <div>
                    <h3 className="font-semibold mb-2">Shipping Info</h3>
                    <form className="space-y-3">
                        <input
                            type="text"
                            placeholder="Full Name"
                            className="w-full border rounded px-3 py-2"
                        />
                        <input
                            type="text"
                            placeholder="Address"
                            className="w-full border rounded px-3 py-2"
                        />
                        <input
                            type="text"
                            placeholder="City"
                            className="w-full border rounded px-3 py-2"
                        />
                        <input
                            type="text"
                            placeholder="ZIP Code"
                            className="w-full border rounded px-3 py-2"
                        />
                    </form>

                    <div className="mt-4 border-t pt-3">
                        <p className="flex justify-between">
                            <span className="font-medium">Total</span>
                            <span>₹{order?.totalAmount}</span>
                        </p>
                    </div>

                    <button
                        onClick={handlePayment}
                        className="mt-4 w-full bg-blue-500 text-white py-2 rounded"
                    >
                        Proceed to Payment
                    </button>
                </div>
            )}

            {step === "payment" && (
                <div className="text-center">
                    <p className="text-gray-600">Processing Payment...</p>
                    <div className="mt-3 animate-spin rounded-full h-10 w-10 border-4 border-blue-500 border-t-transparent mx-auto"></div>
                </div>
            )}

            {step === "success" && (
                <div className="text-center">
                    <h3 className="text-green-600 font-bold text-lg">✅ Payment Successful!</h3>
                    <p className="mt-2 text-gray-600">
                        Thank you for your purchase. Your order will be shipped soon.
                    </p>
                </div>
            )}
        </div>
    );
}

💬Integrate Tool Responses in UI

Your chat page processes three types of messages:

• normal model text

• tool calls

• tool responses

// app/page.tsx
"use client";

import {useChat} from "@ai-sdk/react";
import {DefaultChatTransport} from "ai";
import {useState} from "react";
import {Bot, User} from "lucide-react";
import ProductCard from "@/components/ProductCard";
import Cart from "@/components/CartCard";
import Checkout from "@/components/Checkout";
import {OrderItem} from "@/lib/orderStore";

export default function Home() {
    const {messages, sendMessage, status} = useChat({
        transport: new DefaultChatTransport({
            api: '/api/chat',
        }),
    });
    const [input, setInput] = useState('');

    return (
        <main className="flex flex-col items-center pt-4 min-h-screen bg-white">
            <div className="w-[70vw] bg-white rounded-lg p-6">
                <h1 className="text-2xl font-bold mb-4 text-center">
                    🛍️ Shopping Assistant
                </h1>

                {/* Chat messages */}
                <div className="space-y-4 h-[calc(100vh-200px)] w-full overflow-y-auto p-4 rounded-lg mb-4">
                    {messages.map((m) => (
                        <div
                            key={m.id}
                            className={`flex flex-start items-start gap-3 items-center`}
                        >
                            {m.role === "user" ? (
                                <div className="flex items-center gap-2">
                                    <div className="bg-blue-500 text-white p-2 rounded-full">
                                        <User className="w-5 h-5"/>
                                    </div>
                                </div>) : (
                                <div className="flex items-center gap-2">
                                    <div className="bg-gray-600 text-white p-2 rounded-full">
                                        <Bot className="w-5 h-5"/>
                                    </div>
                                </div>
                            )}
                            <div
                                className={`w-full flex py-[5px] px-[10px] rounded-[10px] ${m.role === "user" ? "bg-blue-100" : "bg-gray-200"}`}>
                                {m.parts.map((part, index) => {
                                        switch (part?.type) {
                                            case 'step-start':
                                                // show step boundaries as horizontal lines:
                                                return index > 0 ? (
                                                    <div key={index} className="text-gray-500">
                                                        <hr className="my-2 border-gray-300"/>
                                                    </div>
                                                ) : null;
                                            case 'tool-fetchCatalog':
                                                switch (part?.state) {
                                                    case 'input-streaming':
                                                        return <span className="italic text-gray-600"
                                                                     key={`tool-${index}`}> 🛍️ Fetching products… </span>;
                                                    case 'input-available':
                                                        return <span className="italic text-gray-600"
                                                                     key={`tool-${index}`}> 🛍️ Fetching products… </span>;
                                                    case 'output-available':
                                                        let products = part.output as any

                                                        return products?.length > 0 ?
                                                           <div className={"flex flex-wrap gap-4"} key={"products-list"}>
                                                                {products?.map((product: any) => {
                                                                    return <ProductCard product={product} addToCart={async (productId:string, productName:string) => {
                                                                        await sendMessage({text: `Add ${productName} to my cart`})
                                                                    }}
                                                                                        key={product.id}/>
                                                                })}
                                                                    </div>
                                                            :
                                                            <div key={`tool-${index}`} className="text-gray-600">No
                                                                products found.</div>
                                                    case 'output-error':
                                                        return <div key={`tool-${index}`}>Error: {part.errorText}</div>;
                                                }
                                                break;
                                            case 'tool-addToCart':
                                                switch (part?.state) {
                                                    case 'input-streaming':
                                                        return <span className="italic text-gray-600"
                                                                     key={`tool-${index}`}>
                                                            ➕ Adding item to cart…
                                                        </span>;
                                                    case 'input-available':
                                                        return <span className="italic text-gray-600"
                                                                     key={`tool-${index}`}>
                                                            ➕ Adding item to cart…
                                                        </span>;
                                                    case 'output-available':
                                                        return <Cart key={`tool-${index}}` } items={part.output as any[]}/>;
                                                    case 'output-error':
                                                        return <div key={`tool-${index}`}>Error: {part.errorText}</div>;
                                                }
                                                break;
                                            case 'tool-checkoutCart':
                                                const order = part?.output as OrderItem
                                                switch (part?.state) {
                                                    case 'input-streaming':
                                                        return <span className="italic text-gray-600"
                                                                     key={`tool-${index}`}>💳 Processing checkout…</span>;
                                                    case 'input-available':
                                                        return <span className="italic text-gray-600"
                                                                     key={`tool-${index}`}>💳 Processing checkout…</span>;
                                                    case 'output-available':
                                                        return <Checkout order={order} key={`tool-${index}`}/>;
                                                    case 'output-error':
                                                        return <div key={`tool-${index}`}>Error: {part.errorText}</div>;
                                                }
                                                break;

                                            case 'text':
                                                return <span key={index}>{part.text}</span>;
                                        }
                                    }
                                )}
                            </div>
                        </div>
                    ))}
                    {status === 'submitted' && <div className="text-gray-500">Thinking...</div>}
                </div>

                {/* Input box */}
                <form onSubmit={e => {
                    e.preventDefault();
                    if (input.trim()) {
                        sendMessage({text: input});
                        setInput('');
                    }
                }} className="flex gap-2">
                    <input
                        className="flex-1 border border-gray-300 shadow-lg rounded-lg px-4 py-2 focus:outline-none focus:ring-1 focus:ring-gray-400"
                        value={input}
                        disabled={status !== 'ready'}
                        placeholder="Ask about products... e.g. 'Show me shoes under 200 rupees'"
                        onChange={e => setInput(e.target.value)}
                    />
                    <button
                        type="submit"
                        className="bg-blue-500 text-white px-4 py-2 rounded-lg hover:bg-blue-600 shadow-lg disabled:opacity-50"
                        disabled={status !== 'ready' || !input.trim()}
                    >
                        Ask
                    </button>
                </form>
            </div>
        </main>
    );
}

Result

List all products

Filter products based on price

Add to cart using text

Checkout feature using text that allows user to pay using preferred method

🎯 What We Just Built

With Gemini tool calling + Vercel AI SDK + Next.js, you now have an:

• AI-powered shopping assistant

• Interactive product catalog chatbot

• Cart-enabled conversational checkout

• Real-time streaming chat UI

• Modern e-commerce experience using React card components

🚀 Next Steps

Add more advanced features:

• 🔍 Vector search (Supabase, Pinecone)

• 🔧 Tool-based filtering (by price, brand, ratings)

• 👤 Personalized recommendations

• 💳 Real checkout integration

• 📦 Track order status

🏁 Conclusion

Using Vercel AI SDK + Google Gemini, we've created a fully interactive e-commerce chatbot that transforms the shopping experience. This innovation allows users to discover products, add items to their cart, and complete the checkout process all within a conversational interface. This development is a significant advancement in the tech world, offering a seamless and engaging way to shop online.

Here is the live version: DEMO

Source code - Github

In this tutorial, we’ll walk through building a simple Ethereum dApp using Next.js, Wagmi, and MetaMask. Our application will let users:

• Connect their MetaMask wallet

• Send ETH to another address on a testnet (Sepolia)

• View their recent transactions in a paginated format

This project is designed as a starting point for anyone who wants to dive into Ethereum development and understand how wallet connections, transactions, and blockchain interactions work in practice.

We’ll use MetaMask as the crypto wallet for connecting and signing transactions, Wagmi (a React hooks library) for interacting with Ethereum, and Next.js for building the frontend of our dApp.

By the end of this guide, you’ll have a fully functional Ethereum dApp running on a testnet—ready to extend into something bigger, like token transfers or DeFi features.

Prerequisites

Before we dive into building the dApp, here’s what you’ll need to get started:

1. Basic Knowledge

• Familiarity with React and Next.js fundamentals

• A basic understanding of Ethereum and how transactions work

• Some exposure to JavaScript/TypeScript

2. Installed Tools

• Node.js & npm/yarn – To run the Next.js project

• MetaMask – Installed as a browser extension or installed on mobile (for connecting and approving transactions)

• VS Code (or any IDE) – To write and manage your project code

3. Ethereum Test Network Setup

We’ll use the Sepolia testnet for this project. That way, you don’t spend real ETH while testing.

👉 Steps:

1. Install MetaMask if you haven’t already.

2. Switch your MetaMask network to Sepolia Test Network.

3. Get some free test ETH from a Sepolia Faucet (just Google Sepolia faucet and request test ETH).

💡 Note: If you’re having trouble getting test ETH, you can also reach out to me via [email], and I can transfer some Sepolia ETH to your wallet for testing.

4. Libraries We’ll Use

• Next.js – React framework for building the frontend

• Wagmi – React hooks for Ethereum (easy wallet connection + transaction handling)

• Viem – A low-level Ethereum library used under the hood by Wagmi

• Tailwind CSS – For styling (optional, but makes UI much easier)

Once you have these prerequisites in place, you’ll be ready to start coding your dApp 🚀

🚀 Project Setup

Let’s set up our Ethereum dApp from scratch. We’ll be using Next.js for the frontend, Wagmi for wallet interactions, and MetaMask as our crypto wallet provider.

1. Create a Next.js App

First, create a new Next.js project using the official CLI:

npx create-next-app eth-dapp
cd eth-dapp

2. Install Dependencies

We’ll need Wagmi, viem, and ethers to connect with Ethereum.

npm install wagmi viem ethers

If you want pretty logging (optional), install pino-pretty:

npm install pino-pretty

3. Configure Wagmi

Inside your project, set up the Wagmi client in a Web3Provider.tsx file. This will allow us to connect to Ethereum testnets like Sepolia.

"use client"
import {createConfig, http, WagmiProvider} from 'wagmi'
import {mainnet, sepolia} from 'wagmi/chains'
import {QueryClient, QueryClientProvider} from "@tanstack/react-query";
import {metaMask} from "@wagmi/connectors";
import {injected} from "wagmi/connectors";

const queryClient = new QueryClient();
export const config = createConfig({
    chains: [mainnet, sepolia],
    connectors: [
        metaMask(),
        injected(),
    ],
    transports: {
        [mainnet.id]: http(),
        [sepolia.id]: http(),
    },
})

export function Web3Provider({children}: { children: React.ReactNode }) {
    return <WagmiProvider config={config}>
        <QueryClientProvider client={queryClient}>
            {children}
        </QueryClientProvider></WagmiProvider>;
}

Now wrap your app in this provider (app/layout.tsx in Next.js 13+):

import {Web3Provider} from "@/lib/web3/Web3Provider";

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
            <Web3Provider>{children}</Web3Provider>
      </body>
    </html>
  );
}

4. Install MetaMask

Make sure you have the MetaMask extension installed in your browser or in your phone.

• Switch to the Sepolia Testnet in MetaMask.

• Request some test ETH from a Sepolia Faucet.

💡 If you’re unable to get test ETH, feel free to email me at [icon.gaurav806@gmail.com] and I can transfer some to your wallet for testing.

👉 That’s our base setup. Next, we’ll build the Wallet Connection component so users can connect their MetaMask wallet to the dApp.

🔗 Connecting Your Wallet

Now that our project is set up with Next.js and Wagmi, let’s add a Wallet Connect button so users can link their MetaMask wallet to the dApp.

1. Create a Wallet Connect Component

Inside components/WalletConnect.tsx, add the following:

"use client"
import { useState } from "react"
import {useAccount, useConnect, useDisconnect} from "wagmi";

export default function WalletConnect({ onAddWallet }: {
    onAddWallet: (address: string) => void
}) {
    const {connectAsync, connectors, status , reset, error} = useConnect();
    const { disconnect } = useDisconnect();
    const handleConnect = async () => {
        try {
            disconnect(); // Disconnect any existing connection
            const connector = connectors[0]; // Assuming the first connector is the one we want
            const data = await connectAsync({ connector });
            onAddWallet(data?.accounts?.[0]);
        } catch (error) {
            reset()
            console.error("Failed to connect wallet:", error);
            console.log(error)
        }
    }


    return (
        <div className="max-w-md mx-auto mt-10 p-6 bg-white rounded-2xl shadow-md">
            <h2 className="text-xl font-semibold text-gray-800 mb-4">Add Wallet</h2>

            <div className="text-center">
                <button
                    onClick={handleConnect}
                    className="w-full py-2 px-4 bg-green-600 text-white rounded-lg hover:bg-green-700 transition"
                >
                    {status === 'pending'? 'Connecting...' :'Connect Using External Wallet'}
                </button>

                {error && <p className="text-red-600 mt-2">{error.message}</p>}
            </div>
        </div>
    )
}

2. Add It to the Homepage

Open app/page.tsx and include the WalletConnect component:

import WalletConnect from "@/components/WalletConnect";

export default function Home() {
  return (
    <div className="flex flex-col items-center justify-center px-6">

            {/* ===== Hero Section with Wallet Connect ===== */}
            <section className="text-center mt-20 max-w-3xl">
                <h1 className="text-5xl font-extrabold text-gray-900 mb-6">
                    Manage Your Crypto with Ease 🚀
                </h1>
                <p className="text-lg text-gray-600 mb-8">
                    Create wallets, send & receive Ethereum, and track transactions —
                    all in one secure, easy-to-use app.
                </p>

                {/* Connect CTA */}
                <div className="mt-6">
                    <WalletConnect onAddWallet={() => redirect('/dashboard')}/>
                    <p className="text-sm text-gray-500 mt-3">
                        Don’t have a wallet yet? Install{" "}
                        <Link
                            href="https://metamask.io/download/"
                            target="_blank"
                            rel="noopener noreferrer"
                            className="text-blue-600 underline"
                        >
                            MetaMask
                        </Link>{" "}
                        to get started.
                    </p>
                </div>
            </section>
    </div>
  );
}

3. Test It

• Run your dev server:

npm run dev

• Open http://localhost:3000.

• 

• Click Connect Wallet → MetaMask should pop up asking for approval.

• 

• Once connected, you’ll see get redirected to dashboard page.

✅ At this stage, your dApp can connect and disconnect a wallet.
Next, we’ll extend this by fetching account details and showing balances.

💰 Fetching Wallet Details & Balances

Once users connect their wallet, we can show them their Ethereum address, balance, and even a quick copy button. This helps them confirm they are on the right account before sending transactions.

1. Fetch Account Details with Wagmi

Wagmi provides hooks like useAccount and useBalance to make this easy. Inside app/dashboard/page.tsx, add the following:

"use client";

import {useAccount, useBalance} from "wagmi";
import {useState} from "react";
import {Copy, Send, Wallet, ExternalLink, X} from "lucide-react";
import {redirect} from "next/navigation";
import SendTransaction from "@/components/SendTransaction";

export default function DashboardPage() {
    const {address, isConnected} = useAccount();
    const {data: balance} = useBalance({address});


    const handleCopy = () => {
        if (address) {
            navigator.clipboard.writeText(address);
            alert("Wallet address copied!");
        }
    };

    if (!isConnected) {
        return (
            <div className="flex items-center justify-center min-h-screen p-4">
                <div className="p-6 border rounded-lg shadow-md text-center w-full max-w-sm">
                    <p className="text-lg font-semibold">No wallet connected</p>
                    <p className="text-sm text-gray-500 mt-2">
                        Please connect your wallet to view your dashboard.
                    </p>
                </div>
            </div>
        );
    }

    return (
        <div className="p-4 sm:p-8 space-y-6 sm:space-y-8">
            <h1 className="text-2xl sm:text-3xl font-bold">Dashboard</h1>

            {/* Wallet Info */}
            <div className="border rounded-lg shadow-md p-4 sm:p-6 flex items-center justify-between gap-4">
                <div className="flex items-center gap-4 w-full">
                    <Wallet className="w-8 h-8 sm:w-10 sm:h-10 text-blue-600"/>
                    <div className="min-w-0 flex-1">
                        <p className="text-sm text-gray-500">Connected Wallet</p>
                        <p className="font-semibold text-sm sm:text-base break-all ">{address}</p>
                    </div>
                </div>
                <button
                    onClick={handleCopy}
                    className="p-2 border rounded-lg hover:bg-gray-100 flex-shrink-0"
                >
                    <Copy className="w-4 h-4"/>
                </button>
            </div>


            {/* Balance Info */}
            <div className="border rounded-lg shadow-md p-4 sm:p-6">
                <p className="text-sm text-gray-500">Total Balance</p>
                <p className="text-xl sm:text-2xl font-bold mt-2">
                    {balance ? `${balance.formatted} ${balance.symbol}` : "Loading..."}
                </p>
            </div>


        </div>
    );
}

2. Test It

• Connect your wallet.

• Head to /dashboard.

• You should now see your wallet address, a copy icon, and your ETH balance.

✅ With this, your dApp shows real-time account info after connection.
Next up, we’ll implement the Send ETH feature with a modal for transfers.

🚀 Sending ETH with a Transaction Modal

One of the key features of our dApp is the ability to send ETH directly from the dashboard. For this, we’ll build a modal form where users can:

• Enter a recipient’s address

• Enter the amount to send

• Confirm the transfer in MetaMask

We’ll use wagmi’s useSendTransaction hook along with viem’s parseEther utility.

1. Send Transaction Component

Inside components/SendTransaction.tsx, add the following:

"use client";

import { useState } from "react";
import { useSendTransaction, useWaitForTransactionReceipt } from "wagmi";
import { parseEther } from "viem";
import Link from "next/link";

export default function SendTransaction() {
    const [to, setTo] = useState<`0x${string}` | "">("");
    const [amount, setAmount] = useState("");

    const { data: txHash, isPending, sendTransaction, error } = useSendTransaction();
    const { isLoading: isConfirming, isSuccess: isConfirmed } =
        useWaitForTransactionReceipt({
            hash: txHash, // hash of transaction
        });

    const handleSend = async () => {
        try {
            await sendTransaction({
                to: to as `0x${string}`,
                value: parseEther(amount), // convert ETH string to wei
            });
        } catch (err) {
            console.error("Transaction failed:", err);
        }
    };

    return (
        <div className="p-4 max-w-md mx-auto space-y-4 rounded">
            <h2 className="text-lg font-semibold">Send ETH</h2>

            <input
                type="text"
                placeholder="Recipient address (0x...)"
                value={to}
                onChange={(e) => setTo(e.target.value as `0x${string}` | "")}
                className="w-full p-2 border rounded"
            />

            <input
                type="text"
                placeholder="Amount (ETH)"
                value={amount}
                onChange={(e) => setAmount(e.target.value)}
                className="w-full p-2 border rounded"
            />

            <button
                onClick={handleSend}
                disabled={isPending}
                className="px-4 py-2 bg-blue-600 text-white rounded hover:bg-blue-700 disabled:bg-gray-400"
            >
                {isPending ? "Sending..." : "Send"}
            </button>

            {/* Transaction States */}
            {isPending && <p className="text-yellow-600">⏳ Transaction is being sent...</p>}
            {isConfirming && <p className="text-blue-600">⏳ Waiting for confirmation...</p>}
            {isConfirmed && (
                <p className="text-green-600">
                    ✅ Transaction confirmed!
                </p>
            )}

            {/* Transaction Details */}
            {txHash && (
                <div className="mt-2 space-y-2">
                    <p className="text-sm break-all">
                        🔗 Tx Hash: {txHash}
                    </p>
                    <Link
                        href={`https://sepolia.etherscan.io/tx/${txHash}`}
                        target="_blank"
                        className="underline text-blue-600"
                    >
                        View on Etherscan
                    </Link>
                    <br />
                    <Link
                        href={`/dashboard`}
                        className="underline text-purple-600"
                    >
                        Go to Dashboard
                    </Link>
                </div>
            )}

            {error && (
                <p className="text-sm text-red-600">
                    ❌ Error: {error.message}
                </p>
            )}
        </div>
    );
}

2. Add Modal to Dashboard

We’ll trigger the modal using a Send Money button.

"use client";

import {useAccount, useBalance} from "wagmi";
import {useState} from "react";
import {Copy, Send, Wallet, ExternalLink, X} from "lucide-react";
import {redirect} from "next/navigation";
import SendTransaction from "@/components/SendTransaction";

export default function DashboardPage() {
    const {address, isConnected} = useAccount();
    const {data: balance} = useBalance({address});

    const [isModalOpen, setIsModalOpen] = useState(false);

    const handleCopy = () => {
        if (address) {
            navigator.clipboard.writeText(address);
            alert("Wallet address copied!");
        }
    };

    if (!isConnected) {
        return (
            <div className="flex items-center justify-center min-h-screen p-4">
                <div className="p-6 border rounded-lg shadow-md text-center w-full max-w-sm">
                    <p className="text-lg font-semibold">No wallet connected</p>
                    <p className="text-sm text-gray-500 mt-2">
                        Please connect your wallet to view your dashboard.
                    </p>
                </div>
            </div>
        );
    }

    return (
        <div className="p-4 sm:p-8 space-y-6 sm:space-y-8">
            <h1 className="text-2xl sm:text-3xl font-bold">Dashboard</h1>

            {/* Wallet Info */}
            <div className="border rounded-lg shadow-md p-4 sm:p-6 flex items-center justify-between gap-4">
                <div className="flex items-center gap-4 w-full">
                    <Wallet className="w-8 h-8 sm:w-10 sm:h-10 text-blue-600"/>
                    <div className="min-w-0 flex-1">
                        <p className="text-sm text-gray-500">Connected Wallet</p>
                        <p className="font-semibold text-sm sm:text-base break-all ">{address}</p>
                    </div>
                </div>
                <button
                    onClick={handleCopy}
                    className="p-2 border rounded-lg hover:bg-gray-100 flex-shrink-0"
                >
                    <Copy className="w-4 h-4"/>
                </button>
            </div>


            {/* Balance Info */}
            <div className="border rounded-lg shadow-md p-4 sm:p-6">
                <p className="text-sm text-gray-500">Total Balance</p>
                <p className="text-xl sm:text-2xl font-bold mt-2">
                    {balance ? `${balance.formatted} ${balance.symbol}` : "Loading..."}
                </p>
            </div>

            {/* Actions */}
            <div className="flex flex-col gap-4 sm:flex-row">
                <button
                    onClick={() => setIsModalOpen(true)}
                    className="flex items-center gap-2 px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 w-full sm:w-auto"
                >
                    <Send className="w-4 h-4"/> Send Money
                </button>

            </div>

            {/* Send Money Modal */}
            {isModalOpen && (
                <div className="fixed inset-0 flex items-center justify-center bg-black/40 p-4">
                    <div className="bg-white rounded-lg shadow-lg w-full max-w-md p-6 relative">
                        <button
                            onClick={() => setIsModalOpen(false)}
                            className="absolute top-2 right-2 p-1 rounded hover:bg-gray-100"
                        >
                            <X className="w-5 h-5"/>
                        </button>
                        <SendTransaction/>
                    </div>
                </div>
            )}
        </div>
    );
}

3. Test the Flow

1. Connect your wallet

2. Enter a recipient address and amount

3. 

   🎉 ETH is transferred on the testnet

4. 

   Confirm the transaction in MetaMask or in Etherscan

✅ Now your dApp supports ETH transfers via a clean modal UI.

Next, we’ll build the Transactions Page to view past transfers with pagination.

📜 Viewing Past Transactions

Once users can send ETH, the next logical step is to view their past transactions. This helps them keep track of transfers, amounts, and recipients directly from our dApp.

We’ll use Etherscan APIs (or any testnet block explorer API) to fetch transaction history for the connected wallet. For simplicity, we’ll show a paginated table of transactions.

1. Transactions Page

Inside app/transactions.tsx, add the following:

"use client";

import {useAccount} from "wagmi";
import {useEffect, useState} from "react";
import {ExternalLink} from "lucide-react";

interface Transaction {
    hash: string;
    from: string;
    to: string;
    value: string;
    timeStamp: number;
}

export default function TransactionsPage() {
    const {address, isConnected} = useAccount();
    const [transactions, setTransactions] = useState<Transaction[]>([]);
    const [page, setPage] = useState(1);
    const [loading, setLoading] = useState(false);

    // replace this with your provider API
    const fetchTransactions = async (pageNumber: number) => {
        if (!address) return;
        setLoading(true);

        try {
            // Example using Etherscan API (you can use Alchemy/Moralis/Blockscout too)
            const res = await fetch(
                `https://api-sepolia.etherscan.io/api?module=account&action=txlist&address=${address}&startblock=0&endblock=99999999&page=${pageNumber}&offset=5&sort=desc&apikey=${process.env.NEXT_PUBLIC_ETHERSCAN_API_KEY}`
            );
            const data = await res.json();
            setTransactions(data.result);

        } catch (err) {
            console.error("Error fetching transactions:", err);
        } finally {
            setLoading(false);
        }
    };

    useEffect(() => {
        if (isConnected) {
            fetchTransactions(page);
        }
    }, [page, isConnected]);

    if (!isConnected) {
        return (
            <div className="flex items-center justify-center min-h-screen">
                <div className="p-6 border rounded-lg shadow-md text-center">
                    <p className="text-lg font-semibold">No wallet connected</p>
                    <p className="text-sm text-gray-500 mt-2">
                        Connect your wallet to view transactions.
                    </p>
                </div>
            </div>
        );
    }

    return (
        <div className="p-8 w-full">
            <h1 className="text-2xl font-bold mb-6">Transactions</h1>

            {loading ? (
                <p>Loading transactions...</p>
            ) : (
                <div className="overflow-x-auto border rounded-lg">
                    <table className="min-w-full text-sm">
                        <thead className="bg-gray-100">
                        <tr>
                            <th className="px-4 py-2 text-left">Sender</th>
                            <th className="px-4 py-2 text-left">Receiver</th>
                            <th className="px-4 py-2">Type</th>
                            <th className="px-4 py-2">Date</th>
                            <th className="px-4 py-2">Amount (ETH)</th>
                            <th className="px-4 py-2">Status</th>
                        </tr>
                        </thead>
                        <tbody>
                        {transactions.map((tx) => {
                            const type =
                                tx.from.toLowerCase() === address?.toLowerCase()
                                    ? "Sent"
                                    : "Received";

                            const date = new Date(
                                parseInt(tx.timeStamp.toString()) * 1000
                            ).toLocaleString();

                            const amount = (parseFloat(tx.value) / 1e18).toFixed(5);

                            return (
                                <tr key={tx.hash} className="border-t hover:bg-gray-50">
                                    <td className="px-4 py-2">{tx.from}</td>
                                    <td className="px-4 py-2">{tx.to}</td>
                                    <td className="px-4 py-2 text-center">{type}</td>
                                    <td className="px-4 py-2">{date}</td>
                                    <td className="px-4 py-2">{amount}</td>
                                    <td className="px-4 py-2 text-center">
                                        <a
                                            href={`https://sepolia.etherscan.io/tx/${tx.hash}`}
                                            target="_blank"
                                            rel="noopener noreferrer"
                                            className="inline-flex items-center gap-1 text-blue-600 hover:underline"
                                        >
                                            View <ExternalLink className="w-4 h-4"/>
                                        </a>
                                    </td>
                                </tr>
                            );
                        })}
                        </tbody>
                    </table>
                </div>
            )}

            {/* Pagination Controls */}
            <div className="flex justify-between mt-4">
                <button
                    disabled={page === 1}
                    onClick={() => setPage((p) => Math.max(p - 1, 1))}
                    className="px-4 py-2 border rounded disabled:opacity-50"
                >
                    Previous
                </button>
                <span className="px-4 py-2">Page {page}</span>
                <button
                    onClick={() => setPage((p) => p + 1)}
                    className="px-4 py-2 border rounded"
                >
                    Next
                </button>
            </div>
        </div>
    );
}

2. Key Features

• Etherscan API integration – fetches real transaction data

• Pagination support – only 5 transactions per page

• Clickable transaction hashes – link to Sepolia Etherscan explorer

• Formatted ETH values & timestamps

3. User Flow

1. User navigates to Transactions Page

2. dApp fetches their latest transactions from Sepolia testnet

3. Users can browse through pages for full history

4. Each transaction is clickable → Opens in Etherscan

In this guide, we built a simple yet powerful Ethereum dApp using Next.js, Wagmi, and MetaMask.
We walked through:

• 🔗 Connecting a crypto wallet with MetaMask

• 👛 Checking wallet balances on the Sepolia testnet

• 💸 Sending ETH transactions directly from the browser

• 📜 Viewing transaction history with pagination and Etherscan links

This project gives you a solid foundation for building decentralized applications. From here, you can expand in many exciting directions:

🚀 Possible Next Steps

• Token Transfers (ERC-20): Add support for sending tokens like USDC or DAI.

• Smart Contracts: Deploy your own contracts and interact with them.

• NFT Support (ERC-721/1155): Mint, transfer, and display NFTs in your dApp.

• Improved UI/UX: Add notifications, confirmations, and a polished dashboard.

• Security Features: Integrate rate limiting, error handling, and input validation.

👉 This dApp is just the starting point. With these building blocks, you can craft anything from a DeFi dashboard to a full-scale Web3 application.

🌐 Try it out yourself
You can explore the live version here: Deployed dApp

💻 View or fork the code
Check out the full source code on GitHub: GitHub Repo

🔮 What’s Next in Our Blog Series?

In the next post, we’ll explore integrating smart contracts into our dApp so that users can interact with decentralized logic directly from the UI.

Instead of rigid endpoints and verbose documentation, MCP offers a dynamic, model-based architecture that’s inherently more compatible with how LLMs operate. In this guide, I’ll show you how to build an MCP server backed by PostgreSQL using FastMCP and psycopg2, explain the pain points it solves and walk you through the key steps to get it running—so you can focus on building smarter systems without drowning in boilerplate. Let’s dive in.

What is an MCP Server?

Model Context Protocol (MCP) is an open-source set of rules, developed by Anthropic, that establishes a standard way for applications to provide relevant context to Large Language Models (LLMs).

Envision MCP as a universal connector, much like a USB-C port for AI. Just as USB-C allows various devices to connect to a wide range of accessories in a standardized manner, MCP enables AI models to seamlessly interface with diverse data sources and tools. This standardization simplifies the process of building AI agents and complex workflows, offering pre-built integrations and the flexibility to work with different LLM providers while prioritizing data security within your own infrastructure.

Without interfaces like MCP, LLMs are limited to their built-in capabilities and training data. With MCP, they can be empowered to:

• Read files and databases

• Execute commands

• Access APIs

• Interact with local tools

• And more!

All of this happens with user oversight and permission, making it both powerful and secure.

MCP Architecture

MCP has the following components

• MCP Hosts: Programs like Claude Desktop, IDEs, or AI tools that want to access data through MCP

• MCP Clients: Protocol clients that maintain 1:1 connections with servers

• MCP Servers: Lightweight programs that each expose specific capabilities through the standardized Model Context Protocol

• Local Data Sources: Your computer’s files, databases, and services that MCP servers can securely access

• Remote Services: External systems available over the internet (e.g., through APIs) that MCP servers can connect to

Core MCP Concepts

MCP servers can provide three main types of capabilities:

1. Resources: File-like data that can be read by clients (like API responses or file contents)

2. Tools: Functions that can be called by the LLM (with user approval)

3. Prompts: Pre-written templates that help users accomplish specific tasks

MCP Resource

Resources are MCP’s way of exposing read-only data to LLMs. A resource is anything that has content that can be read, such as:

• Files on your computer

• Database records

• API responses

• Application data

• System information

Each resource has:

• A unique URI (like [file:///example.txt](file:///example.txt) or database://users/123)

• A display name

• Optional metadata (description, MIME type)

• Content (text or binary data)

MCP Tools

Tools are executable functions that LLMs can call to perform actions or retrieve dynamic information. Unlike resources, which are read-only, and prompts, which structure LLM interactions, tools allow LLMs to actively do things like calculate values, make API calls, or modify data.

Tools enable LLMs to interact with systems and perform actions.

What are MCP Prompts?

Prompts in MCP are structured templates that servers provide to standardize interactions with language models. Unlike resources which provide data, or tools which execute actions, prompts define reusable message sequences and workflows that help guide LLM behavior in consistent, predictable ways.

In this article we are going to create mcp server for postgreSQL step by step

Setting Up Your Development Environment

First, let’s install uv and set up our Python project and environment:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Make sure to restart your terminal afterwards to ensure that the uv command gets picked up.

Create Project Directory

Now, let’s create and set up our project:

# Create a new directory for our project
uv init mcp-server
cd mcp-server

# Create virtual environment and activate it
uv venv
.venv\Scripts\activate

# Install dependencies
uv add mcp[cli] httpx

# Create our server file
new-item server.py

Now let’s dive into building your server.

Building your server

Update the server.py according to the following code:

1. Import the packages

2. Database wrapper class for database connection and database queries

3. Define App Context

4. Define app lifespan and pass this lifespan to MCP instance along with MCP Server name

from contextlib import asynccontextmanager
from collections.abc import AsyncIterator
from dataclasses import dataclass

import asyncpg
from mcp.server.fastmcp import FastMCP, Context

# Async PostgreSQL wrapper using asyncpg
class Database:
    def __init__(self, pool: asyncpg.Pool):
        self.pool = pool

    @classmethod
    async def connect(cls) -> "Database":
        pool = await asyncpg.create_pool(
            user="#user",
            password="#your_password",
            database="Your_database_name",
            host="#database_host",
            port='#Port_number',
        )
        return cls(pool)

    async def disconnect(self):
        await self.pool.close()

    async def query(self, query: str) -> list:
        async with self.pool.acquire() as conn:
            try:
                return await conn.fetch(query)
            except Exception as e:
                print(f"Query error: {e}")
                return []

    async def fetch_schema(self) -> list:
        query = """
        SELECT table_name FROM information_schema.tables
        WHERE table_schema = 'public'
        """
        return await self.query(query)


@dataclass
class AppContext:
    db: Database


@asynccontextmanager
async def app_lifespan(server: FastMCP) -> AsyncIterator[AppContext]:
    db = await Database.connect()
    try:
        yield AppContext(db=db)
    finally:
        await db.disconnect()


mcp = FastMCP("PostgresMCPServer", lifespan=app_lifespan)

The FastMCP class uses Python type hints and docstrings to automatically generate tool definitions, making it easy to create and maintain MCP tools.

Implementing MCP tools

The tool execution handler is responsible for actually executing the logic of each tool. Let’s add it:

@mcp.tool("fetch_schema")
async def fetch_schema(ctx: Context) -> str:
    db = ctx.request_context.lifespan_context.db
    schema = await db.fetch_schema()
    return str([record["table_name"] for record in schema])


@mcp.tool("fetch_all_tables")
async def fetch_all_tables(ctx: Context) -> str:
    db = ctx.request_context.lifespan_context.db
    query = "SELECT * FROM information_schema.tables WHERE table_schema='public'"
    tables = await db.query(query)
    return str(tables)


@mcp.tool("run_query")
async def run_query(ctx: Context, query: str) -> str:
    db = ctx.request_context.lifespan_context.db
    result = await db.query(query)
    return str(result)

​Running the server

Finally, let’s initialize and run the server:

if __name__ == "__main__":
    # Initialize and run the server
    mcp.run()

Your server is complete! Run uv run server.py to confirm that everything’s working.

Let’s now test your server from an existing MCP host, Claude for Desktop.

Testing your server with Claude for Desktop

There are several ways to test your MCP server. One way is with Claude Desktop, and you can also use the MCP Inspector tool to test all capabilities during development.

Setting Up Claude Desktop

Here are the steps for setting up an MCP server in Claude Desktop:

1. Install Claude for Desktop if you haven’t already

2. Open Claude and access Settings

   You can access it in your [path of claude desktop installation]/claude_desktop_config.json

3. Edit configuration

    {
      "mcpServers": {
        "PostgresMCPServer": {
          "command": "uv",
          "args": [
            "run",
            "--with",
            "mcp[cli]",
            "mcp",
            "run",
            "C:\\ABSOLUTE\\PATH\\TO\\PARENT\\FOLDER\\server.py"
          ]
        }
      }
    }
   

This tells Claude for Desktop:

1. There’s an MCP server named “weather”

2. To launch it by running uv command with the following arguments

Save the file, and restart Claude for Desktop.

Test with Claude Desktop

Let’s make sure Claude for Desktop is picking up the 3 tools we’ve exposed in our PostgresMCPServer server. You can do this by looking for the setting icon

After clicking on the setting icon, you should see the MCP server listed:

After clicking the PostgresMCPServer, you should see the MCP tools listed:

If you can see the tools here, you can now test your server

We will test the server using the following commands:

1. Retrieve information on all tables within the database.

2. Use Claude Desktop to summarize the relationships between the tables.

3. Use Claude to generate a bar graph illustrating the comparison between events and registration counts.

Finally…

Building an MCP server powered by PostgreSQL offers a dynamic and efficient way to integrate large language models with structured data. By leveraging the Model Context Protocol, developers can create adaptive systems that bypass the limitations of traditional APIs, allowing AI agents to interact seamlessly with various data sources and tools. This approach not only simplifies the development process but also enhances the capabilities of AI models, enabling them to perform complex tasks with greater ease and security. By following the steps outlined in this guide, you can set up your own MCP server and unlock new possibilities for smarter, more responsive systems.

Enter GraphQL Federation—a powerful solution that lets you compose a unified GraphQL API from multiple microservices, while keeping each service independently developed and deployed.

In this article, we’ll break down what GraphQL Federation is, how it works, and why it’s a game-changer for modern backend systems.

🧩 What Is GraphQL Federation?

GraphQL Federation is a technique introduced by Apollo that allows multiple GraphQL services (called subgraphs) to be merged into a single, unified API gateway (called the federated gateway).

It solves a critical issue in microservice architectures: how to let teams work independently on their own GraphQL schemas, while still offering a seamless client experience through a single graph.

Key Components:

🔍 Core Concepts Explained

1. Entity Resolution with @key

If multiple services work on the same entity (e.g., User), the @key directive tells the gateway how to resolve it:

type User @key(fields: "id") {
  id: ID!
  name: String
}

2. Service Extension with @extends

A service can add fields to an entity defined in another service:

# In the reviews subgraph
extend type User @key(fields: "id") {
  id: ID! 
  reviews: [Review]
}

3. Gateway Composition

The gateway uses service definitions from each subgraph (via introspection or static configs) and composes them into a single schema that the client can query.

💡 Real-World Example: E-Commerce Platform

Let’s say you're building an e-commerce platform with the following services:

• User Service: Manages user data.

• Product Service: Manages products available in the store.

• Order Service: Manages customer orders and ties users to the products they purchase.

With GraphQL Federation, each service defines its part of the schema and can extend entities from other services to build a unified graph.

🔹 Schema in User Service:

type User @key(fields: "id") {
  id: ID!
  name: String
  email: String
}

This defines the User entity and its primary key (id). It can be referenced by other services.

🔹 Schema in Product Service:

type Product @key(fields: "id") {
  id: ID!
  name: String
  price: Float
}

Each product has its own ID, name, and price.

🔹 Schema in Order Service:

type Order @key(fields: "id") {
   id: ID!
   quantity: Int
   orderDate: String
   user:User @provides(fields: "id")
   product:Product @provides(fields: "id")
   total: Float
}

extend type User @key(fields: "id") {
  id: ID! 
}

extend type Product @key(fields: "id") {
  id: ID! 
}

This service defines the Order entity, and extends both User and Product entities to show which users placed orders and which products were purchased.

🧪 Unified Query at the Gateway:

Once the services are federated into a single gateway, clients can query them as one unified schema:

query {
  orders {
    id
    quantity
    orderDate
    user {
      id
      name
      email
    }
    product {
      id
      name
      price
    }
  }
}

This single query fetches user details, their orders, and associated product details—even though the data is spread across three separate services.

🚦 How Federation Makes This Work

• The User Service owns the User type.

• The Product Service owns the Product type.

• The Order Service stitches everything together by referencing User and Product entities using the @extends directive.

This setup allows each team to focus on their domain, deploy independently, and still contribute to a shared graph that feels seamless to the client.

⚙️ Setting Up Apollo Federation (Simplified Steps)

To federate your services using Apollo Federation, follow these steps:

🔧 Step 1: Set Up Each Subgraph (User, Product, Order)

Each service is an independent GraphQL server using the @apollo/subgraph package.

1.1 Install Dependencies

npm install @apollo/subgraph graphql

👤 User Service

Schema (user-schema.graphql)

type User @key(fields: "id") {
  id: ID!
  name: String
  email: String
}

Server Setup (index.js)

const { ApolloServer } = require('@apollo/server');
const { buildSubgraphSchema } = require('@apollo/subgraph');
const { startStandaloneServer } = require('@apollo/server/standalone');
const gql = require('graphql-tag');

const typeDefs = gql(require('fs').readFileSync('./user-schema.graphql', 'utf-8'));
const resolvers = {
  User: {
    __resolveReference(user) {
      return users.find(u => u.id === user.id);
    },
  },
};

const server = new ApolloServer({
  schema: buildSubgraphSchema([{ typeDefs, resolvers }]),
});

startStandaloneServer(server, { listen: { port: 4001 } });

📦 Product Service

Schema (product-schema.graphql)

type Product @key(fields: "id") {
  id: ID!
  name: String
  price: Float
}

Server Setup (index.js)

Use the same @apollo/subgraph setup, listening on port 4002.

📑 Order Service

This one extends both User and Product entities.

Schema (order-schema.graphql)

type Order @key(fields: "id") {
  id: ID!
  user: User
  products: [Product]
  total: Float
}

extend type User @key(fields: "id") {
  id: ID!
  orders: [Order]
}

extend type Product @key(fields: "id") {
  id: ID!
  purchasedBy: [User]
}

Server Setup

Use the same @apollo/subgraph setup, listening on port 4003.

🛠 Step 2: Set Up Apollo Gateway

This service composes the subgraphs and exposes one unified schema.

Install Required Packages

npm install @apollo/gateway graphql @apollo/server

Gateway Setup (gateway.js)

const { ApolloServer } = require('@apollo/server');
const { ApolloGateway } = require('@apollo/gateway');
const { startStandaloneServer } = require('@apollo/server/standalone');

const gateway = new ApolloGateway({
  serviceList: [
    { name: 'user', url: 'http://localhost:4001' },
    { name: 'product', url: 'http://localhost:4002' },
    { name: 'order', url: 'http://localhost:4003' },
  ],
});

const server = new ApolloServer({ gateway, subscriptions: false });

startStandaloneServer(server, { listen: { port: 4000 } }).then(({ url }) => {
  console.log(`🚀 Gateway ready at ${url}`);
});

🔁 Step 3: Start All Services

In separate terminal tabs or scripts:

# Terminal 1
node user/index.js

# Terminal 2
node product/index.js

# Terminal 3
node order/index.js

# Terminal 4
node gateway.js

Now your GraphQL API is live at http://localhost:4000/graphql and unified!

🧪 Step 4: Query the Federated Schema

Test it with a powerful, nested query:

query {
  orders {
    id
    quantity
    orderDate
    user {
      id
      name
      email
    }
    product {
      id
      name
      price
    }
  }
}

Even though user, order, and product are handled by separate microservices, this single query works flawlessly via federation!

⚖️ Pros and Cons of Using GraphQL Federation

Before adopting GraphQL Federation in your architecture, it’s important to weigh its advantages and potential challenges. Here’s a balanced look:

✅ Pros of GraphQL Federation

⚠️ Cons of GraphQL Federation

🧠 When to Use GraphQL Federation

Use Federation when:

• You have multiple teams working on different domains (e.g., user, orders, products).

• You want to avoid monolithic GraphQL servers.

• You’re already on a microservices architecture and need a clean API layer.

• You need to enable schema composition and controlled entity extension.

Avoid Federation when:

• You have a small team or a small monolithic app.

• Your microservices don't share much schema or are loosely coupled.

• You’re not familiar with GraphQL or don’t have time to invest in tooling/setup.

🎯 Conclusion: Future-Proof Your Backend

GraphQL Federation brings structure, scalability, and clarity to the chaos of microservice APIs. It aligns perfectly with the modular development goals of backend teams, while keeping the developer experience smooth on both server and client ends.

If you're building a microservices architecture or already using GraphQL, federation is worth exploring—especially with tools like Apollo Federation leading the way.

👉 Ready to federate your GraphQL APIs?
Start small—break one schema into subgraphs and try running a gateway locally. You’ll see the power of Federation in action.

Don’t forget to share this post with your team and bookmark it for reference!

DataLoader is a generic utility developed by Facebook for batching and caching database queries efficiently in GraphQL applications. It helps in reducing redundant queries and solving the N+1 query problem by grouping multiple queries into a single batch request.

Key Features of DataLoader:

1. Batching: Combines multiple database requests into a single query to optimize performance.

2. Caching: Stores results to prevent redundant database calls in the same request cycle.

3. Asynchronous Execution: Uses promises to handle multiple database requests efficiently.

By integrating DataLoader into GraphQL resolvers, we can significantly improve the efficiency and scalability of our APIs.

Why DataLoader is Necessary 🧐

GraphQL APIs provide flexibility in data fetching, allowing clients to request only the necessary data. However, this flexibility can lead to the N+1 query problem, a common performance issue where multiple queries to related data cause database inefficiencies.

The N+1 Query Problem 🏗️

In a GraphQL resolver, if fetching related data requires separate queries for each parent record, it results in an exponential increase in database calls. Consider an example:

const resolvers = {
  Query: {
    users: async () => await db.User.findAll(),
  },
  User: {
    posts: async (parent) => await db.Post.findAll({ where: { userId: parent.id } })
  }
};

If we fetch 10 users along with their posts, the resolver first queries for users (1 query), then for each user, it fetches their posts (10 additional queries). This results in 11 queries instead of an optimal 2 queries.

How DataLoader Solves This Problem

DataLoader batches multiple queries into a single request and caches results to avoid redundant calls. It allows GraphQL resolvers to efficiently fetch related data in bulk. 📦🔗⚡

Step-by-Step Implementation📝

1. Install DataLoader 🔧

If not already installed, add DataLoader to your project:

npm install dataloader

2. Create a DataLoader for Batching Queries 📊

In your GraphQL setup, define a DataLoader instance to batch and cache database queries.

const DataLoader = require('dataloader');
const db = require('./models');

const userPostLoader = new DataLoader(async (userIds) => {
  const posts = await db.Post.findAll({ where: { userId: userIds } });

  const postsByUserId = userIds.map(id => posts.filter(post => post.userId === id));
  return postsByUserId;
});

3. Integrate DataLoader in Resolvers 🔌

Modify the resolver to use DataLoader instead of making separate queries.

const resolvers = {
  Query: {
    users: async () => await db.User.findAll(),
  },
  User: {
    posts: (parent, args, context) => context.userPostLoader.load(parent.id)
  }
};

4. Attach DataLoader to Context 🔗

Ensure DataLoader is available in each request by adding it to the context in your GraphQL server setup.

const { ApolloServer } = require('apollo-server');

const server = new ApolloServer({
  typeDefs,
  resolvers,
  context: () => ({
    userPostLoader: userPostLoader
  })
});

Verifying Performance Improvements 📊

Before implementing DataLoader, let's analyze the number of queries being made. Consider the following GraphQL query:

query {
  users {
    id
    name
    posts {
      id
      title
    }
  }
}

Without DataLoader : ❌

1. Query to fetch users (1 query)

    SELECT * FROM users;
   

2. Query for each user's posts (10 queries if there are 10 users)

    SELECT * FROM posts WHERE userId = 1;
    SELECT * FROM posts WHERE userId = 2;
    ...
    SELECT * FROM posts WHERE userId = 10;
   

Total queries: 11

With DataLoader : ✅

1. Query to fetch users (1 query)

    SELECT * FROM users;
   

2. Single batched query to fetch all posts at once (1 query)

    SELECT * FROM posts WHERE userId IN (1,2,3,4,5,6,7,8,9,10);
   

Total queries: 2

Performance Gains : 🚀

By reducing the number of queries from 11 to 2, DataLoader significantly reduces database load and improves response time. The reduction in queries is more noticeable as the dataset grows, ensuring better scalability and performance efficiency. Before DataLoader, fetching posts for 10 users resulted in 11 queries. With DataLoader, it reduces to 2 queries:

1. Fetch all users

2. Fetch all posts in a single batch query

Integrating DataLoader with Prisma/Sequelize/MongoDB 🛠️

Using DataLoader with Prisma

const userPostLoader = new DataLoader(async (userIds) => {
  const posts = await prisma.post.findMany({
    where: { userId: { in: userIds } },
  });

  const postsByUserId = userIds.map(id => posts.filter(post => post.userId === id));
  return postsByUserId;
});

Using DataLoader with Sequelize

const userPostLoader = new DataLoader(async (userIds) => {
  const posts = await Post.findAll({
    where: { userId: userIds },
  });

  return userIds.map(id => posts.filter(post => post.userId === id));
});

Using DataLoader with MongoDB (Mongoose)

const userPostLoader = new DataLoader(async (userIds) => {
  const posts = await Post.find({ userId: { $in: userIds } });

  return userIds.map(id => posts.filter(post => post.userId.toString() === id.toString()));
});

Impact of Using DataLoader

1. Improved Performance: Reduces database queries, significantly optimizing response times.

2. Better Scalability: Handles large GraphQL queries efficiently, making the API more scalable.

3. Reduced Database Load: Batching reduces the number of queries, minimizing database stress.

4. Caching Benefits: Avoids redundant database calls by caching previously fetched data.

Conclusion 💡

Using DataLoader in GraphQL APIs is essential for optimizing database queries and solving the N+1 query problem. By batching requests and caching results, it significantly enhances performance and scalability. Whether you are using Prisma, Sequelize, or MongoDB, integrating DataLoader is a best practice for efficient GraphQL APIs.

Enter GraphQL plugins—a powerful mechanism in Apollo Server that allows developers to customize and enhance their API functionality with ease.

In this blog, we’ll explore what GraphQL plugins are, set up a basic Apollo Server, and walk through creating a custom plugin to log query response times. We’ll also delve into advanced use cases and best practices for building scalable and efficient GraphQL APIs with Apollo Server plugins. Let’s dive in!

What Are GraphQL Plugins?

GraphQL plugins in Apollo Server allow developers to inject custom logic into the server’s lifecycle. Whether it’s tracking performance, managing request authentication, or adding additional logging, plugins empower developers to fine-tune their GraphQL servers for specific needs.

Lifecycle Hooks

Plugins in Apollo Server operate on a series of lifecycle hooks. These hooks allow developers to tap into various stages of a GraphQL operation, such as:

1. requestDidStart: Triggered when a new GraphQL request is received.

2. didResolveOperation: Triggered after the server successfully parses and validates the query.

3. executionDidStart: Triggered before the query execution begins.

4. willSendResponse: Triggered right before the server sends the response to the client.

By leveraging these hooks, you can implement custom behaviors like query logging, error tracking, or request throttling.

The following diagram illustrates the sequence of events that fire for each request. Each of these events is documented in Apollo Server plugin events and available on apollo website.

Setting Up Apollo Server

Before we jump into creating a custom plugin, let’s set up a basic Apollo Server with a simple schema and resolver. If you don’t already have Apollo Server installed, start by setting it up:

Step 1: Install Dependencies

Run the following command to install the necessary packages:

npm install @apollo/server graphql

Step 2: Define a Basic Schema

Create a file named schema.js with the following content:

// we store posts list in a separate file
import {posts} from "../../utils/data"

const typeDefs = `#graphql
    type Query {
        posts:[Post]
    }

    type Post {
        id: ID!
        title: String!
        content: String!
    }
`;

const resolvers = {
  Query: {
    posts: () => {
            return posts;
        },
  },
};

module.exports = { typeDefs, resolvers };

Step 3: Set Up the Apollo Server

Create a file named server.js and set up Apollo Server:

const { ApolloServer } = require('@apollo/server');
const { typeDefs, resolvers } = require('./schema');

const server = new ApolloServer({
  typeDefs,
  resolvers,
});

server.listen().then(({ url }) => {
  console.log(`🚀 Server ready at ${url}`);
});

Run the server using the following command:

node server.js

At this point, you have a basic Apollo Server running locally.

Creating a Custom Plugin for Response Time Logging

Now that we have a working Apollo Server, let’s create a custom plugin to log the response time of each GraphQL query.

Step 1: Understanding the Plugin Lifecycle

To measure response time, we’ll use the requestDidStart and willSendResponse hooks. Here’s the flow:

1. Record the start time when the request begins.

2. Calculate the duration when the response is about to be sent.

Step 2: Implementing the Plugin

Create a file named loggingPlugin.js with the following content:

const loggingPlugin = {
  async requestDidStart(requestContext) {
    const start = Date.now();
    console.log(`Query received: ${requestContext.request.query}`);

    return {
      async willSendResponse() {
        const duration = Date.now() - start;
        console.log(`Response sent after ${duration}ms`);
      },
    };
  },
};

module.exports = { loggingPlugin };

Step 3: Integrating the Plugin

Modify your server.js file to include the custom plugin:

const { ApolloServer } = require('@apollo/server');
const { typeDefs, resolvers } = require('./schema');
const { loggingPlugin } = require('./loggingPlugin');

const server = new ApolloServer({
  typeDefs,
  resolvers,
  plugins: [loggingPlugin],
});

server.listen().then(({ url }) => {
  console.log(`🚀 Server ready at ${url}`);
});

Restart the server, and you’ll see query logs along with response times in the console.

Advanced Use Cases for Apollo Server Plugins

Plugins offer immense flexibility, enabling you to build advanced features for your GraphQL APIs. Here are a few examples:

1. Extending the Logging Plugin

Enhance the plugin to include user-specific details or operation names in the logs. For instance:

console.log(`Operation: ${requestContext.operationName}`);
console.log(`User ID: ${requestContext.context.userId}`);

2. Enforcing Security and Rate Limiting

Implement rate limiting or request validation to prevent abuse. For example:

• Track the number of queries.

• Reject requests exceeding a predefined threshold.

if (queryCount > MAX_QUERIES) {
  throw new Error('Rate limit exceeded.');
}

3. Monitoring and Analytics

Send query performance metrics to external tools like Prometheus, Grafana, or DataDog for detailed monitoring and analytics.

4. Dynamic Schema Modifications

Build a plugin that dynamically modifies the schema at runtime based on user roles or feature flags.

Conclusion

Apollo Server plugins are a powerful tool for enhancing the functionality and performance of your GraphQL APIs. From logging response times to enforcing security and enabling advanced monitoring, plugins offer endless possibilities to tailor your server to your application’s needs.

By following this guide, you now have the skills to create custom plugins and integrate them seamlessly into your Apollo Server. Experiment with different use cases and share your innovations with the community.

For further learning, check out the Apollo Server documentation. Happy coding! 🚀

If you found this guide helpful, share it with your network and let us know how you’re using Apollo Server plugins in your projects. Stay tuned for more tutorials on building scalable GraphQL APIs!

Essential Elements for Ensuring API Reliability

1. Performance Monitoring

   • Ensure APIs respond quickly and handle concurrent requests efficiently.

   • Optimize for low latency and high throughput.

2. Scalability

   • Design APIs to handle increasing loads without degradation.

   • Use load balancers, auto-scaling groups, and caching mechanisms.

3. Error Handling

   • Implement comprehensive error logging and monitoring.

   • Provide clear error responses for better client-side debugging.

4. Security

   • Secure APIs with authentication, authorization, and encryption.

   • Monitor for unusual activity or unauthorized access attempts.

5. Availability

   • Aim for high uptime with robust failover mechanisms.

   • Use redundancy at the network, server, and data levels.

Key Metrics to Monitor for API Success

1. Performance Metrics

   

   • Response Time (Latency): Average, 95th percentile (P95), and 99th percentile (P99) latencies.

   • Request Rate (RPS): Total requests per second handled by the API.

   • Error Rate: Percentage of requests resulting in errors (4xx or 5xx responses).

   • Cache Hit Rate: Frequency of cache hits versus total cache requests.

2. Infrastructure Metrics

   • CPU and Memory Usage: Resource consumption patterns under load.

   • Disk I/O: Storage throughput for reading and writing data.

   • Thread and Connection Pool Usage: Health of connection pools and threads.

3. Reliability Metrics

   • Uptime: Measure of API availability, often reflected as an SLA.

   • Dependency Latency: Response time for third-party APIs the service relies on.

   • Timeouts and Retries: Frequency of timed-out requests and retries.

4. Usage Metrics

   

   • Endpoint Popularity: Most accessed API endpoints.

   • User Activity Patterns: Trends in API usage over time.

   • Rate Limit Violations: Incidents where clients exceed allowed limits.

5. Security Metrics

   • Authentication Failures: Invalid login attempts or token issues.

   • Unusual IP Activity: Unexpected access patterns from specific IPs.

   • Data Integrity Issues: Monitoring anomalies in data processing or storage.

How to Monitor These Metrics

These tools provide complete solutions for monitoring performance, resource use, and reliability metrics.

• Datadog: Comprehensive monitoring and alerting with integrated APM and logging.

• New Relic: APM with powerful diagnostics and distributed tracing.

• AWS CloudWatch: Built-in monitoring for AWS-based infrastructure and APIs.

These tools stand out due to their wide adoption, robust features, and the ability to address various aspects of API monitoring, from performance metrics to log analysis.

Best Practices for API Monitoring

1. Use Distributed Tracing: Tools like Jaeger or Zipkin help trace requests across services, identifying bottlenecks.

2. Implement Logging: Use structured logging to capture detailed request/response data for troubleshooting.

3. Automate Alerts: Set up alerts for anomalies like high error rates, increased latency, or resource exhaustion.

4. Conduct Regular Load Testing: Use tools like Apache JMeter or k6 to simulate traffic and identify scaling issues.

5. Continuously Refine Metrics: Regularly review and update monitored metrics to align with evolving business needs.

Conclusion

In conclusion, ensuring API reliability is a multifaceted endeavor that requires careful attention to performance, scalability, error handling, security, and availability. By monitoring key metrics such as response time, error rate, and resource usage, developers can gain valuable insights into the health and performance of their APIs. Utilizing powerful monitoring tools like Prometheus, Datadog, and AWS CloudWatch can aid in effectively tracking these metrics and identifying potential issues before they impact users. Adopting best practices such as distributed tracing, structured logging, and regular load testing further enhances the robustness of APIs. By implementing these strategies, developers can significantly improve application performance, leading to a more reliable and satisfying user experience.

Postman is a robust API development environment that enables you to create, send, test, and document APIs efficiently. GitHub Actions is a CI/CD platform designed to automate the testing and deployment of your code. By integrating Postman with GitHub Actions, you can seamlessly automate your API testing as part of your development workflow.

Prerequisites:

• A GitHub account and repository.

• A Postman collection containing your API tests.

• A Postman environment containing the necessary variables for your tests (e.g., API keys, base URLs).

Steps:

1. Create a Postman Collection:

   • In Postman, create a new collection and add your API test cases.

   • Configure the necessary environment variables in your Postman environment.

2. Generate Postman API Key:

   • Go to the settings page and then API keys to generate a new API key

   • Save this API key to GitHub repository secrets

3. Create a GitHub Actions Workflow:

   • In your GitHub repository, create a new file named .github/workflows/postman-tests.yml.

   • Paste the following code into the file, replacing the placeholders with your specific values:

       name: Automated Testing using Postman CLI
     
       on:
         push:
           branches: [main]
     
       jobs:
         automated-api-tests:
           runs-on: ubuntu-latest
           steps:
             - uses: actions/checkout@v4
             - name: Install Postman CLI
               run: |
                 curl -o- "https://dl-cli.pstmn.io/install/linux64.sh" | sh
             - name: Login to Postman CLI
               run: postman login --with-api-key ${{ secrets.POSTMAN_API_KEY }}
             - name: Run API tests
               run: |
                 postman collection run "[collection_id]" -e "[environment_id]"
     

4. Commit and Push:

   • Commit and push the .github/workflows/postman-tests.yml file to your GitHub repository.

Explanation:

• The workflow is triggered by pushing events to the main branch.

• It installs the Postman CLI for Linux

• It uses the postman login command to authenticate with the Postman CLI

• It runs the Postman collection specified by the [collection_id] placeholder (replace it with your actual collection ID). It also uses the environment defined by the [environment_id] placeholder (replace it with your actual environment ID) for variables.

Additional Considerations:

• You can customize the workflow to run tests on different branches, trigger on pull requests, or use different runners.

• Consider using environment variables to store sensitive information (e.g., API keys) securely.

• Explore other features of GitHub Actions, such as caching, artifacts, and secrets, to enhance your workflow.

By following these steps, you can effectively automate the testing of your APIs using Postman and GitHub Actions, ensuring that your code changes are thoroughly tested before deployment.

What is a webhook?

A webhook is essentially an HTTP callback triggered by a specific event in a software application. When a defined event happens, the source system sends an HTTP POST payload to a pre-defined URL, notifying your application about the occurrence.

Why use webhooks?

Webhooks offer several advantages over traditional polling methods:

• Real-time updates: Receive information instantly when events happen, eliminating delays.

• Reduced server load: No need for constant polling, saving resources.

• Efficient resource utilization: Trigger actions only when necessary, based on specific events.

Real-world examples:

• E-commerce platforms sending order confirmation or shipment updates to your application

• Payment gateways notifying you about successful or failed transactions

• Collaboration tools sending notifications about file changes or comments

How webhooks work:

1. Event triggering: An event occurs in the source system (e.g., order placed, payment processed).

2. Webhook request: The source system sends an HTTP POST request to the pre-defined webhook URL.

3. Payload delivery: The request includes data about the event in the request body (often in JSON format).

4. Webhook endpoint: Your application receives the request and processes the data accordingly.

By understanding the fundamentals of webhooks, we're ready to dive into building a basic webhook service in Node.js.

Setting Up Your Node.js Project for Webhooks

To get started, we'll need a Node.js project. Use npm init -y to quickly create a new project directory. Next, install the required dependencies:

npm install express body-parser cors

• Express.js: For building the web server.

• body-parser: For parsing incoming request bodies.

• cors: For handling Cross-Origin Resource Sharing (CORS) if needed.

Now the next steps include creating CRUD operations to store webhooks that need to be triggered when an event happened

Implementing CRUD Operations for Webhook Storage

1. Setup MongoDB instance using Mongoose: Create a file named db.js to connect to your MongoDB database:

    const mongoose = require('mongoose');
   
    const connectDB = async () => {
      try {
        await mongoose.connect('mongodb://localhost:27017/webhook_service', {
          useNewUrlParser: true,
          useUnifiedTopology: true,
        });
        console.log('MongoDB connected successfully');
      } catch (error) {
        console.error('MongoDB connection error:', error);
        process.exit(1); // Exit process on connection failure
      }
    };
   
    module.exports = connectDB;
   

2. Define Webhook schema :

   Create a file named Webhook.js to define the Mongoose model for your webhook data:

    const mongoose = require('mongoose');
   
    const WebhookSchema = new mongoose.Schema({
      url: {
        type: String,
        required: true,
      },
      headers: {
        type: Object,
        default: {},
      },
      events: {
        type: [String], // Array of strings representing subscribed events
        required: true,
      },
      createdAt: {
        type: Date,
        default: Date.now,
      },
     // Additional considerations:
   
      secret: {
        type: String,
        default: '', // Optional secret key for authentication
      },
      isActive: {
        type: Boolean,
        default: true, // Flag indicating if the webhook is currently active
      },
      description: {
        type: String,
        default: '', // Optional description of the webhook's purpose
      },
   
    });
   
    module.exports = mongoose.model('Webhook', WebhookSchema);
   

3. Define endpoint to list all webhooks

    // Read (GET) all webhooks
    app.get('/webhooks', async (req, res) => {
      try {
        const webhooks = await Webhook.find();
        res.json(webhooks);
      } catch (error) {
        console.error(error);
        res.status(500).send('Server Error');
      }
    });
   

4. Define endpoints to store a webhook in the database

    // Create (POST) a new webhook
    app.post('/webhooks', async (req, res) => {
      try {
        const newWebhook = new Webhook(req.body);
        const savedWebhook = await newWebhook.save();
        res.status(201).json(savedWebhook);
      } catch (error) {
        console.error(error);
        res.status(500).send('Server Error');
      }
    });
   

5. Define an endpoint to fetch a single webhook using the id

    // Read (GET) a single webhook by ID
    app.get('/webhooks/:id', async (req, res) => {
      try {
        const webhook = await Webhook.findById(req.params.id);
        if (!webhook) {
          return res.status(404).send('Webhook not found');
        }
        res.json(webhook);
      } catch (error) {
        console.error(error);
        res.status(500).send('Server Error');
      }
    });
   

6. Define an endpoint to update a webhook using id

    // Update (PUT) a webhook by ID
    app.put('/webhooks/:id', async (req, res) => {
      try {
        const updatedWebhook = await Webhook.findByIdAndUpdate(
          req.params.id,
          req.body,
          { new: true } // Return the updated document
        );
        if (!updatedWebhook) {
          return res.status(404).send('Webhook not found');
        }
        res.json(updatedWebhook);
      } catch (error) {
        console.error(error);
        res.status(500).send('Server Error');
      }
    });
   

7. Define an endpoint to delete a webhook using id

    // Delete (DELETE) a webhook by ID
    app.delete('/webhooks/:id', async (req, res) => {
      try {
        const deletedWebhook = await Webhook.findByIdAndDelete(req.params.id);
        if (!deletedWebhook) {
          return res.status(404).send('Webhook not found');
        }
        res.json({ message: 'Webhook deleted successfully' });
      } catch (error) {
        console.error(error);
        res.status(500).send('Server Error');
      }
    });
   

Creating Webhook Triggering Logic in Node.js

In this section, we will create an endpoint that generates a dummy event and subsequently triggers the webhooks subscribed to this event.
The steps included in this logic are:

1. Fetch all webhooks that are subscribed to the particular event

2. Define the payload for each webhook

3. Send a POST request to each webhook with the respective payload

app.post('/generate-event', async (req, res) => {
  try {
        const {event, data} = req.body;

        // fetch all webhooks that subscribed to this event
        const webhooks = await Webhook.find({
            events:event
        });



        // Define webhook payload
        const webhookPayload = {
            event: event,
            data: data,
        };

        // Send POST request to each webhook endpoint
        for (const webhook of webhooks) {
            await axios.post(webhook?.url, webhookPayload);
        }


        res.status(200).json({ message: 'Event generated and webhook triggered successfully' });
    } catch (error) {
        console.error('Error generating event and triggering webhook:', error);
        res.status(500).json({ error: 'Internal server error' });
    }
});

Building a Webhook Receiver Step-by-Step

A webhook receiver is essentially a listening service. It's like a mailbox where other applications can send you messages. Here in our example, we define an endpoint /user-webhook that will receive data, process it and log it on screen.

• Define a new route: In your index.js file, create a new route /user-webhook to handle incoming webhook requests.

• Access the request body: Use req.body to access the data sent in the webhook payload.

• Log the webhook payload: It's helpful to log the received webhook data for debugging purposes:

• Send a successful response: After processing the webhook, send a successful HTTP response to the sender. This indicates that our server has received and processed the webhook successfully.

app.post('/webhook', (req, res) => {
  console.log('Webhook received:', req.body);
  res.status(200).send('Webhook received');
});

Testing Your Webhook Service - Practical Use Cases

To test out our webhook service we have to break the tests into 3 use cases as shown below

1. Store the webhook details: Use a tool like Postman to send a POST request to your endpoint to store webhook details.

   

2. Trigger the event: Create a dummy event and trigger the webhook.

   

3. Check results on our receiver endpoint: Verify that the payload is received at the specified endpoint.

   

This is a basic example. In a production environment, you'll likely need to implement more robust error handling, validation, and asynchronous processing for webhooks.

Securing the Webhook Endpoint

Security is paramount when handling webhooks. Here are some essential measures:

• IP Whitelisting: Restrict incoming requests to specific IP addresses to prevent unauthorized access.

• API Keys or Tokens: A secret token or API key is required to be included in the request headers for authentication.

• HTTPS: Use HTTPS to encrypt data transmission.

• Content Verification: Verify the integrity of incoming webhook data using checksums or digital signatures.

• Rate Limiting: Implement rate limiting to protect against abuse and denial-of-service attacks.

Validating and Processing Incoming Webhook Data

Before processing webhook data, it's essential to validate it for correctness and security:

• Data Format Validation: Ensure the incoming data adheres to the expected format (JSON, XML, etc.).

• Data Integrity Verification: Check for missing or invalid fields.

• Data Type Validation: Verify that data types match the expected schema.

• Security Checks: Look for potential malicious content or attacks.

Once validated, process the webhook data based on its content. This might involve storing the data, triggering other actions, or updating the application state.

Best Practices for Webhook Implementation

• Reliable Delivery: Implement retry mechanisms and dead letter queues for failed deliveries.

• Error Handling: Provide informative error messages and log errors for debugging.

• Scalability: Design your webhook service to handle increasing traffic and data volumes.

• Security: Prioritize security measures like authentication, authorization, and data encryption.

• Documentation: Create clear documentation for developers using your webhook service.

In conclusion, building a secure and efficient webhook service in Node.js involves understanding the fundamentals of webhooks, setting up a Node.js project, implementing CRUD operations, and creating robust webhook triggering and receiving mechanisms. By following best practices such as securing endpoints, validating incoming data, and ensuring reliable delivery, you can create a resilient and scalable webhook service. This guide provides a comprehensive roadmap to help you achieve real-time updates and efficient resource utilization, ultimately enhancing the performance and security of your application.

What is Database sharding?

Database sharding is a technique for horizontal scaling of databases, where the data is split across multiple database instances, or shards, to improve performance and reduce the impact of large amounts of data on a single database. (ref: Database Sharding)

Each shard is essentially a separate database instance or server that contains a subset of the overall data. The goal of sharding is to distribute the data and database load across multiple shards, allowing for improved scalability, performance, and fault tolerance in a distributed system.

Key components of sharding:

1. Shard Key: This is the attribute used to distribute data across different shards. It's like a label that determines which shard a particular piece of data belongs to. Choosing the right shard key is crucial for even data distribution and efficient querying. Common shard keys include user ID, product category, or date.

2. Shards: These are individual databases that hold a subset of the overall data. Imagine a large library divided into multiple sections - each section (shard) focuses on a specific category of books. The total data is distributed across these shards based on the shard key.

3. Shard Map (or Catalog): This acts like a directory, keeping track of which shard stores data for a specific shard key value. When your application needs to access data, it consults the shard map to locate the appropriate shard. Think of it as a library card catalogue that tells you which section (shard) to find a particular book (data) based on its title or author (shard key).

4. Shard Router: This is a component within your application or a separate service that interacts with the shard map. It receives the shard key from your application logic and uses the shard map to determine the appropriate shard for the desired operation (read or write). The shard router then directs the request to the specific shard.

Benefits of Sharding

• Enhanced Scalability: Sharding allows for horizontal scaling by distributing data across multiple databases (shards). As data volume increases, additional shards can be seamlessly integrated, ensuring the application can accommodate growth without performance sacrifices.

• Improved Performance: Sharding optimizes query execution by enabling queries to target specific data subsets within individual shards. This reduces the amount of data scanned, resulting in significantly faster response times and a more responsive user experience.

• Increased Availability: Sharding introduces a degree of fault tolerance. If one shard encounters an issue, other shards remain unaffected, mitigating the impact on overall system availability. This enhances application resilience and ensures service continuity.

Implementation in Node.js

Now that we have a foundational understanding of database sharding and its benefits, let's dive into a practical implementation example using Node.js and MongoDB.

Create Two Shards (Database Instances)

In this example, we have created two database instances, both hosted on Atlas. You can also use Docker or self-host if you prefer.

Define the shard map

To define the shard map, create a simple JavaScript object that maps product categories to their respective shard connection strings.

const shardMap = {
    'Clothing': 'database url for clothing products',
    'Electronics': 'database url for electronics products',
    'default': 'database url for all other products'

}

Define the function that returns the proper shard based on the category

Create a function that takes a product category as input and returns the appropriate shard connection.

// Fetch the right shard
const fetchShard = (category) => {
    // get shard credentials
    const credentials = shardMap?.[category]
    // connect with the server
    return new MongoClient(credentials ?? shardMap?.default);
}

Define the API routes for uploading products

Set up an API route to handle product uploads. This route will use the fetchShard function to determine the correct shard for storing the product.

app.post('/products', async (req, res) => {

    const {products} = req.body;
    for (const product of products) {
        // get database connection for each product
        const client = await fetchShard(product?.category)
        const db = client.db('database-sharding')
        const productCollection = db.collection("products");
        await productCollection.insertOne(product)
    }
    res.send("ok")
})

Define the API route that returns products based on category

Set up an API route to fetch products by category. This route will also use the fetchShard function to determine the correct shard to query.

app.get('/products/:category', async (req, res) => {
    const {category} = req.params;
    try {
        // find out the correct shard
        const client = await fetchShard(category)
        // connect to the database
        const db = client.db('database-sharding')
        const productCollection = db.collection("products");
        const products = await productCollection.find({category}).toArray();
        res.send(products)
    } catch (err) {
        console.log(err)
        res.status(500).json({message: 'Error fetching products'});
    }
});

Test out our application

To test the application, you can use tools like Postman or Curl to send HTTP requests to the API endpoints. For example:

1. To upload products:

   

2. To fetch products by category:

   

   

Considerations and Best Practices for Sharding

Database sharding offers significant advantages in terms of scalability and performance, but it's important to be aware of its complexities and best practices for successful implementation:

• Increased Application Complexity: Sharding introduces an additional layer of abstraction between your application and the database. You'll need to manage shard routing, shard key selection, and potential consistency issues across shards.

• Shard Key Selection: Choosing the right shard key is critical. It should evenly distribute data across shards and align with your most frequent query patterns. A poorly chosen shard key can lead to bottlenecks and negate the benefits of sharding.

• Scalability Considerations: While sharding enables horizontal scaling, it's not a magic bullet. Adding new shards comes with its own management overhead. Evaluate your data access patterns and growth projections to determine if sharding is the most suitable solution for your specific needs.

By understanding the benefits and considerations of sharding, you can make informed decisions to optimize your Node.js applications for scalability and performance as your data demands continue to evolve. Remember, sharding is a powerful tool, but like any powerful tool, it requires careful planning and execution to reap the maximum rewards.

Understanding Message Queues

Imagine a conveyor belt, producers place tasks on the belt (queue), and separate worker processes (consumers) pick them up for execution. This asynchronous approach decouples producers and consumers, allowing them to operate independently without hindering each other's performance. Message queues offer significant advantages:

• Scalability: Effortlessly scale producers and consumers to manage increasing workloads.

• Resilience: Tasks persist in the queue, ensuring delivery even if producers or consumers encounter temporary outages.

• Asynchronous Processing: Producers don't have to wait for tasks to finish processing, leading to a more responsive application.

• Fault Tolerance: Queues automatically retry tasks upon failures, enhancing system reliability

Getting Started with BullMQ

1. Installation:

   Begin by installing the bullmq and ioredis packages using npm:

    npm install --save bullmq ioredis
   

   bullmq provides core functionalities for managing queues and jobs, while ioredis acts as the Redis client library.

2. Connecting to Redis:

   We are going to use a self-hosted Redis server, download redis from here and install it in your system https://redis.io/downloads/

    const Redis = require('ioredis');
   
    const redisConfig = 
   
    const redisConnection = new Redis({
      port: 6379, // Replace with your Redis port if different
      host: '127.0.0.1', // Replace with your Redis host if different
    });
   
    module.exports = redisConnection;
   

   This code defines the Redis configuration and creates a connection object.

3. Creating a Queue:

   Import the Queue class from BullMQ and your Redis connection in your main application file (e.g., index.js)

    import {Queue} from 'bullmq';
   
    const commentNotificationQueue = new Queue('comment-email-notification', 
    {connection: redisConnection});
   

   Here, you create a new BullMQ queue named comment-email-notification. The connection property specifies the Redis connection established earlier.

Adding Jobs to the Queue

1. Job Data:

   Jobs in BullMQ consist of data you want to process asynchronously. This data can be anything relevant to your task, such as file paths, user IDs, or API request parameters.

2. Adding a Job:

   Use the add method on your queue instance to add a new job:

    async function addJobs() {
        const comments = [
            {
                "text": "I really enjoyed your article on machine learning! The explanation of different algorithms was very clear and concise. Keep up the good work!",
                "commenter_name": "John Smith",
                "commenter_email": "john.smith@example.com", "author_name": "Jane Doe",
                "author_email": "jane.doe@example.com"
            },
            {
                "text": "This phone is amazing! The battery life is fantastic, and the camera takes stunning photos. I would highly recommend it to anyone looking for a new phone.",
                "commenter_name": "Sarah Lee",
                "commenter_email": "sarah.lee@example.com", "author_name": "Stephan Josh"
            },
            {
                "text": "Thanks for your insights on the future of AI. I agree that ethical considerations are crucial as this technology advances.",
                "commenter_name": "David Kim",
                "commenter_email": "david.kim@example.com", "author_name": "Michael Brown",
                "author_email": "michael.brown@example.com"
            }
        ]
        for (const comment of comments) {
            await commentNotificationQueue.add(comment?.commenter_name, comment);
        }
    }
   
    await addJobs();
   

   This code adds a job named based on commenter's name with the specified data (jobData) to the comment-email-notification queue.

3. Job Options (Optional):

   BullMQ offers various options for customizing job behaviour:

   • delay: Schedule the job for execution later (in milliseconds).

   • attempts: Define the number of retries for failed jobs.

   • backoff: Set a backoff strategy for retries after failures (e.g., exponential delay).

   • priority: Assign a priority value to jobs (higher values are processed sooner).

These options provide granular control over job processing and error handling.

This is how the jobs are stored in the Redis database:

Consuming Jobs (Workers)

1. Worker Creation:

   Workers are Node.js processes that continuously listen for new jobs in a queue and execute them. Create a worker using the Worker class:

    const commentNotificationWorker = new Worker('comment-email-notification',
        async job => {
            const {commenter_name, author_name} = job?.data;
            // process comment and send email to author's email about the new comment that he/she receives
   
            console.log(`${commenter_name} commented on ${author_name} article`)
        }, {connection: redisConnection});
   

   This code defines a worker for the comment-email-notification queue. The provided asynchronous function (async (job) => {...}) will be executed for each job received by the worker. Remember to replace the placeholder comment with your actual business logic that utilizes the job.data property.

2. Test the worker:

   This is the output of the worker when the job is executed from the queue

   

   Here is the link to the source code: https://github.com/icon-gaurav/nodejs-bullmq-implementation.git

What are WebSockets?

WebSockets are communication protocols that establish a persistent, two-way connection between a web browser (client) and a web server. Unlike HTTP, which operates on a request-response basis, WebSockets create an open channel for continuous data exchange in both directions – just like in our Google Docs example.

Think of it like a constant conversation:

• HTTP: Imagine having to raise your hand and wait for your turn to speak every time you want to communicate something in Google Docs. This is similar to HTTP, where the client makes a request, waits for the server's response, and then makes another request if needed.

• WebSockets: With WebSockets, it's like having a dedicated open line. You can speak (send data) and hear (receive data) simultaneously, fostering a real-time exchange.

WebSockets are great for building real-time functionality into web applications, such as games and chat apps, or for communicating financial trading data or IoT sensor data.

In this article, let’s create a WebSocket server, and use Postman to send and receive messages across the WebSocket connection.

Setup the project

Create a new directory named websockets

mkdir websockets

Initialize the project using npm

npm init -y

Install the WebSocket library in the project

npm install ws --save

Create a file name server.js in the root directory

import { WebSocketServer } from 'ws';

const wss = new WebSocketServer({ port: 8080 });

wss.on('connection', function connection(ws) {
    ws.on('message', function message(data) {
        console.log('received: %s', data);
    });

    ws.send('Hey!, Welcome to the websocket server!');
});

Run the server

node server.js

Test the server

To test our WebSocket server we will be using Postman.

In Postman, select New > WebSocket Request to open a new tab. Enter the WebSocket server URL. A WebSocket URL begins with ws:// or wss:// and our server is running on localhost:8080. Click Connect.

Once the connection between the server and the postman is established, the Message pane will display the list of incoming and outgoing network messages.

Inspect handshake details.

The connection we established between the Postman client and the local server is bidirectional, which means that in addition to receiving messages, we can also send them. Under the Message tab, write your message and click Send.

We may check the outgoing message in the Messages pane.

Verify the outgoing message was received by your local server:

Send system information

Let's update our WebSocket server and use it to send computer information in a set of interval

Let's install the system information package

npm install systeminformation

Let's add the function that will send a message about the current load on our CPU.

import si from "systeminformation";

 setInterval(async () => {
    const cpuTemp = JSON.stringify(await si.currentLoad());
    ws.send(cpuTemp);
  }, 1000);

Here is the fully updated server.js file

import { WebSocketServer } from 'ws';
import si from "systeminformation";

const wss = new WebSocketServer({ port: 8080 });

wss.on('connection', function connection(ws) {
    ws.on('message', function message(data) {
        console.log('received: %s', data);
    });

    ws.send('Hey!, Welcome to the websocket server!');
    setInterval(async () => {
        const cpuTemp = JSON.stringify(await si.currentLoad());
        ws.send(cpuTemp);
    }, 1000);
});

Save your changes, and restart the local server from the command line:

node server.js

Test the system information

Return to Postman and Connect to the local server. This time, we receive information about our CPU every second!

We have explored the core concepts and walked through building a basic server that sends CPU load time to the client. It's just the beginning, we can leverage WebSockets to build applications that unleash the power of real-time communication like chat applications, real-time notifications etc.

Github source code - https://github.com/icon-gaurav/node-websockets.git

Why SaaS?

The traditional model of software deployment often comes with hefty upfront costs, maintenance headaches, and scalability limitations. This is where Software as a Service (SaaS) shines. SaaS offers a compelling alternative, delivering software applications over the internet in a subscription-based model. This reduces upfront expenses and provides seamless scalability, automatic updates, and enhanced accessibility.

Here's why SaaS is rapidly becoming the go-to choice for businesses of all sizes:

• Reduced Costs: SaaS eliminates the upfront investment required for traditional software licenses and eliminates the need for expensive hardware and IT infrastructure. Subscription fees are typically lower, and the provider handles maintenance costs.

• Scalability: SaaS applications can easily scale up or down based on your needs. No more worrying about overspending on unused software or struggling with insufficient capacity during growth spurts.

• Automatic Updates: SaaS providers handle updates and security patches, ensuring you're always using the latest version with improved features and security fixes.

• Accessibility: SaaS applications are accessible from anywhere with an internet connection and a compatible device. This empowers your workforce with remote working capabilities and improved collaboration.

• Ease of Use: SaaS solutions are typically web-based and user-friendly, requiring minimal training for your team. This reduces the burden on your IT department and gets your team up and running quickly.

Building Your SaaS Empire: Architectural Considerations

Now that we understand the benefits of SaaS, let's delve into the world of building your own SaaS application. The architecture you choose forms the foundation of your application, impacting scalability, security, and performance. Here are the key architectural approaches for SaaS

1. Multi-tenant Architecture:

The most common and cost-effective approach. A single codebase and infrastructure serve multiple tenants (customers). Data isolation techniques like schema isolation or database sharding ensure tenant data remains separate and secure.

Benefits:

• Cost efficiency: Resource sharing reduces infrastructure and maintenance costs.

• Scalable: Easily accommodates a growing user base.

• Faster Development and Deployment: New features are rolled out to all tenants simultaneously.

Challenges:

• Data Isolation Complexity: Guaranteeing complete data separation requires meticulous design and implementation.

• Upgrades and Maintenance: Rollouts can affect all tenants simultaneously.

2. Single-tenant Architecture:

Each tenant has its dedicated application instance and infrastructure. This offers the highest level of customization and control but comes at a higher cost.

Benefits:

• Unparalleled Data Isolation: Each tenant has complete control over its data, eliminating data breach concerns.

• Customization: Tenants can tailor the application to their specific needs.

Challenges:

• Increased Cost: Maintaining separate instances is resource-intensive.

• Slower Development and Deployment: Changes need individual deployment for each tenant instance.

• Scalability: Scaling involves adding resources for each new tenant.

3. Hybrid Architecture:

A blend of multi-tenant and single-tenant approaches. The core application logic might be multi-tenant, while specific components or data can be single-tenant for enhanced customization.

Benefits:

• Cost-Customization Balance: Provides a balance between affordability and tenant-specific needs.

• Flexibility: Allows for tailored solutions based on individual tenant requirements.

Challenges:

• Increased Complexity: Managing this architecture requires more planning and coordination.

• Potential Data Isolation Concerns: Careful design is crucial for single-tenant components.

Choosing the Right Architectural Fit

While designing the architecture for a SaaS application, several factors must be carefully considered to meet the unique requirements of the business and its users

• Number of Tenants: Multi-tenant is generally preferable for a large user base where customization is not of much concern.

• Customization Needs: Single-tenant caters better for highly customized requirements.

• Budget and Scalability Goals: Cost and scalability needs should be carefully considered.

Additional Considerations:

• Microservices Architecture: Breaking down the application into independent, smaller services improves scalability and maintainability. This approach can be implemented with any of the above architectures.

• API-driven Architecture: Exposing functionalities through APIs allows easier integration with third-party applications and services.

By understanding the "whys" and "hows" of SaaS architecture, we can make informed decisions for building a robust and scalable SaaS application that caters to specific business needs. Remember, the chosen architecture is pivotal to your SaaS application's success. Choose wisely!

We can add real-time updates to our graphql API using graphql subscriptions. In this article, we will focus on implementing Subscriptions type that helps in building real-time notification.

What are graphql subscriptions?

Subscriptions are a GraphQL feature that allows a server to send data to its clients when a specific event happens. Subscriptions are just part of your GraphQL contract, and they refer to events. To be able to send these events in real-time, you need to choose a transport that has support for that.
Because the server usually pushes subscription updates (instead of polling by the client), they generally use the WebSocket protocol instead of HTTP.

Setup graphql server

To set up a graphql server for subscription we have to use express middleware because it will run a web socket server and a graphql server. We already know how to set up a graphql server using standalone server configuration (here is the link to that article setup graphql server). So in this section, we only discuss the changes needed for express compatibility. The only thing that needs to change is our server.ts file.

Here is the GitHub link: https://github.com/icon-gaurav/mastering-graphql-with-nodejs/tree/graphql-subscriptions

You can clone this and follow along with my explanation.

import { ApolloServer } from '@apollo/server';
import { expressMiddleware } from '@apollo/server/express4';
import { ApolloServerPluginDrainHttpServer } from '@apollo/server/plugin/drainHttpServer';
import { createServer } from 'http';
import express from 'express';
import { makeExecutableSchema } from '@graphql-tools/schema';
import cors from 'cors';
import resolvers from './graphql/resolvers'
import typeDefs from "./graphql/schema";
import mongoose from "mongoose";

async function startServer(){
    const app = express();

    const httpServer = createServer(app);

    const schema = makeExecutableSchema({
        resolvers,
        typeDefs,
    });

    const apolloServer = new ApolloServer({
        schema,
        introspection:true,
    });

    await apolloServer.start();

    app.use('/graphql', cors<cors.CorsRequest>(), express.json(), expressMiddleware(apolloServer));

    const DATABASE_URL: string = `mongodb://localhost:27017/test`;
    mongoose.connect(DATABASE_URL)
        .then(() => {
            console.log('Database connected successfully')
        })
        .catch((e: any) => {
            console.log('Error connecting : ', e?.message)
        })

    const PORT = 4000;
// Now that our HTTP server is fully set up, we can listen to it.
    httpServer.listen(PORT, () => {
        console.log(`Server is now running on http://localhost:${PORT}/graphql`);
    });
}

startServer();

To start the server run this command npm run start:dev

The server will start at port 4000 using express middleware.

Define subscription in the schema

Schema's Subscription type defines the top-level fields that a user/client can subscribe to. Defining a Subscription type is as easy as defining the schema of type Query and Mutation.

# Subscription type definition in GraphQL Schema
type Subscription {
        postCreated: Post
    }

# Return type of the Subscription
type Post {
        _id: ID!
        title: String!
        content: String!
    }

The postCreated field will get updated whenever a new post is created in the backend and it will notify all clients with the newly created post who are subscribed to this event.

Enable subscriptions

To enable subscriptions we have to start the WebSocket server, here are the easy steps to start the GraphQL Websocket Server

1. Install graphql-ws, ws, and @graphql-tools/schema

    npm install --save graphql-ws ws @graphql-tools/schema
   

2. Create a WebsocketServer instance to use as a subscription server

    import { WebSocketServer } from 'ws';
    import { useServer } from 'graphql-ws/lib/use/ws';
   
    // Creating the WebSocket server
        const wsServer = new WebSocketServer({
            // This is the `httpServer` we created in a previous step.
            server: httpServer,
            // Pass a different path here if app.use
            // serves expressMiddleware at a different path
            path: '/subscriptions',
        });
   

3. Add a plugin to the ApolloServer constructor to clean WebSocketServer and HTTP server, it helps in the proper shutdown of all the servers running on the machine.

   
        const serverCleanup = useServer({ schema }, wsServer);
   
        const apolloServer = new ApolloServer({
            schema,
            introspection:true,
            plugins: [
                // Proper shutdown for the HTTP server.
                ApolloServerPluginDrainHttpServer({ httpServer }),
   
                // Proper shutdown for the WebSocket server.
                {
                    async serverWillStart() {
                        return {
                            async drainServer() {
                                await serverCleanup.dispose();
                            },
                        };
                    },
                },
            ],
        });
   

Here is the fully configured server.ts file.

import { ApolloServer } from '@apollo/server';
import { expressMiddleware } from '@apollo/server/express4';
import { ApolloServerPluginDrainHttpServer } from '@apollo/server/plugin/drainHttpServer';
import { createServer } from 'http';
import express from 'express';
import { makeExecutableSchema } from '@graphql-tools/schema';
import { WebSocketServer } from 'ws';
import cors from 'cors';
import { useServer } from 'graphql-ws/lib/use/ws';
import resolvers from './graphql/resolvers'
import typeDefs from "./graphql/schema";
import mongoose from "mongoose";

async function startServer(){
    const app = express();

    const httpServer = createServer(app);

    const schema = makeExecutableSchema({
        resolvers,
        typeDefs,
    });

    // Creating the WebSocket server
    const wsServer = new WebSocketServer({
        // This is the `httpServer` we created in a previous step.
        server: httpServer,
        // Pass a different path here if app.use
        // serves expressMiddleware at a different path
        path: '/subscriptions',
    });

// Hand in the schema we just created and have the
// WebSocketServer start listening.
    const serverCleanup = useServer({ schema }, wsServer);

    const apolloServer = new ApolloServer({
        schema,
        introspection:true,
        plugins: [
            // Proper shutdown for the HTTP server.
            ApolloServerPluginDrainHttpServer({ httpServer }),

            // Proper shutdown for the WebSocket server.
            {
                async serverWillStart() {
                    return {
                        async drainServer() {
                            await serverCleanup.dispose();
                        },
                    };
                },
            },
        ],
    });

    await apolloServer.start();

    app.use('/graphql', cors<cors.CorsRequest>(), express.json(), expressMiddleware(apolloServer));

    const DATABASE_URL: string = `mongodb://localhost:27017/test`
    mongoose.connect(DATABASE_URL)
        .then(() => {
            console.log('Database connected successfully')
        })
        .catch((e: any) => {
            console.log('Error connecting : ', e?.message)
        })

    const PORT = 4000;
// Now that our HTTP server is fully set up, we can listen to it.
    httpServer.listen(PORT, () => {
        console.log(`Server is now running on http://localhost:${PORT}/graphql`);
    });
}

startServer();

Implement the resolvers for subscription

Resolvers for subscriptions are different from Mutation and Query types, as subscription field resolvers are objects that define a subscribe function. Let's implement a resolver for postCreated subscription. Here is how we define the resolver for the subscription type

Subscription:{
        postCreated: {
            subscribe: () => pubsub.asyncIterator(['POST_CREATED']),
        },
    }

1. The subscribe function must return an object of type AsyncIterator, a standard interface for iterating over asynchronous results.

2. AsyncIterator is generated by pubsub.asyncIterator

3. pubsub.asyncIterator object listens for events that are associated with a label and adds them to a queue to process them

Publish an event

This step involves publishing an event for which we need a real-time notification. This generally happens when there is a new database entry or some updates from the admin side.

In this example, we will publish an "POST_CREATED" event whenever a new post is created using createPost mutation.

createPost: async (_parent: any, args: any, _context: any) => {
            // create new post document
            const post = new Post(args);
            //save post document and return the saved document
            await post.save();
            await pubsub.publish('POST_CREATED', {postCreated: post});
            return post;
        }

pubsub.publish function do the publishing of the event along with the payload of this event which in this case is post data.

Here is the full example of all the resolvers that is needed

import {posts, users, users_posts} from "../utils/data";
import Post from "../models/Post";
import User from "../models/User";
import bcrypt from 'bcrypt';
import jwt from 'jsonwebtoken';
import { PubSub } from 'graphql-subscriptions';

const pubsub = new PubSub();


const resolvers = {
    Query: {
        posts: async (_parent: any, _args: any, context: any) => {
            return Post.find();
        },
    },
    Mutation: {
        // Mutation that will create a new post and save in DB
        createPost: async (_parent: any, args: any, _context: any) => {
            // create new post document
            const post = new Post(args);
            //save post document and return the saved document
            await post.save();
            // publish the "POST_CREATED" event
            await pubsub.publish('POST_CREATED', {postCreated: post});
            return post;
        }
    },
    Subscription:{
        postCreated: {
            // resolving subscribe function
            subscribe: () => pubsub.asyncIterator(['POST_CREATED']),
        },
    }
}

export default resolvers;

Now the server is ready, we can test our subscription.

Test our API

Run the server using npm run start:dev command

Here is the output of this server in which one window defines how we create a post and in another window, we listen to the subscription and see the data updated there. In the same way, we can use this system to build a full-fledged notification system, including authentication, authorization and filtering.

GraphQL subscriptions offer a powerful solution for real-time data updates in web applications. By leveraging subscriptions, developers can enhance user experience by providing instant updates without the need for constant polling. Implementing GraphQL subscriptions is quite fun and I believe we enjoyed it.

Here is the GitHub link: https://github.com/icon-gaurav/mastering-graphql-with-nodejs/tree/graphql-subscriptions

Note: It is not recommended to use the PubSub class for production as it uses an in-memory event publishing system, instead we can use an external datastore-based library according to the use cases.

You can check the library list and further info here: https://www.apollographql.com/docs/apollo-server/data/subscriptions/#production-pubsub-libraries

You can fork this GitHub repository (Repository Link) and get along with me step-by-step

Let's dive in and unlock the secrets to effective GraphQL API testing together!

Installing Jest and required dependencies

# for typescript
npm install --save-dev ts-jest

# for javascript
npm install --save-dev jest

Configuring Jest for testing

Since we are using typescript, we need to configure our application a little bit

Let's initialize the jest using the command

npx ts-jest config:init

This will generate a jest.config.js file. This is what makes jest to work with typescript.

Writing Test cases

Let's create a file named createPost.test.ts in the tests** directory.

Then import the @jest/globals library, which has the required functions to test.

import {describe, expect, test} from '@jest/globals';
import {GraphQLClient} from 'graphql-request';

We need a graphql client to execute the graphql query and get the response to compare the expected result. We are going to use the 'graphql-request' library for that

import {GraphQLClient} from 'graphql-request';

// create a new graphql client for requesting to our api
const client = new GraphQLClient("http://localhost:4000/")

Let's define what we are going to test in this example.

In this example, we are going to test the createPost mutation in which we first create the post using our API and then check whether it has the same title as the post or not

describe('create post mutation', () => {

    test('creates a new post', async () => {

        // graphql query
        const query = `#graphql 
        mutation CREATE_POST_TEST($title: String!, $content: String) {
            createPost(title: $title, content: $content) {
                _id
                content
                content
                title
            }
        }
        `;

        // query variables
        const variables = {
            "title": "testing post title",
            "content": "testing post content"
        };

        let data: any = {};

        try {
            // execute the query
            data = await client.request(query, variables);

        } catch (e) {
            // report the error
            console.log(e)
        }

        expect(data?.createPost).toHaveProperty("title", variables?.title)

    });

});

In the above code

1. We use describe to describe a unit test suite. Even though it's not required, it's helpful to understand what this test suite is all about.

2. In the test, we write the test code. The first argument of this function tells about what this test case is all about and the second argument is a callback function that contains the test code

3. In the callback function first, we sent the API request and then compared the actual and expected responses. The test passes if both match otherwise it fails.

Execute the test cases

There are two steps in executing the test cases

1. Start the API server

npm run start:dev

2. Run jest to execute all of the test cases

jest

Output

With Jest's help, we've learned how to set up a killer testing environment, write top-notch test cases, and integrate testing seamlessly into our workflow. So, let's keep that testing momentum going and deliver some seriously reliable and resilient APIs to our users! Ready to level up? Let's do this!