
I’ve been building products and tools for years — mostly iOS, lots of scripting, some backend, a lot of “make it work and ship it.” Recently I started poking into the AI space. Not just “call an LLM and magic happens,” but the *internals*. I want to understand enough to eventually build something serious on top of this tech.

One of the first things I kept hearing about in RAG (retrieval-augmented generation) setups was **chunking** and **embeddings** — and how they all tie into a **vector database**. I figured I’d document my beginner mental model.

## Step 1: Chunking
This part made sense quickly. You have long documents, but your embedding model (and later the LLM) has token limits. So you split the text into smaller **chunks**.

- Typical chunk size: ~500–800 tokens (2,000–3,200 characters in English)  
- Overlap: 10–20% so context flows between chunks  
- Too big = you get noisy, unfocused vectors. Too small = you lose context.

I now think of chunking as just a preprocessing step to keep everything bite-sized for the next phase.

## Step 2: Embeddings
Once you have chunks, you pass each one through an **embedding model**. This is where my “OK, but how?” curiosity kicked in.

The model outputs a **vector** — literally just an array of floats, like: [0.0213, -0.1342, 0.0876, …]

The magic is that similar meanings end up with vectors that are close together in this high-dimensional space. “Car” and “Automobile” will be neighbors. “Car” and “Banana” will be far apart.

It’s nothing like Levenshtein distance (string edits). Instead, embeddings are trained so semantic closeness translates to geometric closeness.

## Step 3: Storing in a Vector DB
Each chunk’s vector goes into a **row** in the vector database, alongside the original text and metadata. The DB builds an index so later we can search fast.

The vector DB doesn’t care about tokens or text anymore — it’s just arrays of numbers and math.

## Step 4: Retrieval
Here’s the clever bit:

1. User asks a question.  
2. You embed the question into its own vector (same model).  
3. The vector DB finds the **nearest neighbors** to that query vector.  
4. You fetch the corresponding chunks and feed them into your LLM for the final answer.

The naive way is to just scan all rows and pick the highest cosine similarity score (cosine similarity = dot product normalized for vector length). That works for small data, but with millions of vectors it’s too slow.

## Step 5: How Lookup is Actually Fast
Most vector DBs use an **approximate nearest neighbor** index — HNSW is the popular one. My mental model:

- Imagine all your points as cities on a map.  
- The DB builds a set of “highway” layers to jump across the space quickly, then “street” layers to refine locally.  
- When searching, it hops across highways, drops down to streets, and zeroes in on the closest points without visiting every “city.”

This gets you *almost* the same result as scanning everything, but way faster.

## Dot Product & Cosine Similarity (The Core Math)
The dot product is just multiplying each vector component, summing it up.  
Cosine similarity is the dot product divided by the product of vector lengths — which basically strips out magnitude and keeps only the **angle** between the vectors.

For embeddings, angle matters more than length, because we care about **direction in semantic space**, not absolute size.

## Final Mental Picture
- Chunk → embed → store in vector DB → embed query → find nearest vectors → send chunks to LLM.  
- Embeddings are just numbers.  
- Vector search is just “find points closest to this one” with clever shortcuts.  
- The rest is plumbing and product thinking.

I’m only scratching the surface, but already this feels a lot less like magic and more like a set of tools I can reason about.

Learning How Chunking, Embeddings, and Vector Databases Actually Work


As a (bored? topic for another post) startup founder currently between ventures, I've been methodically evaluating AI development tools to understand their practical applications and limitations. After extensive testing, particularly with Cursor and Claude Code, the results vary significantly across different technology stacks.

## The iOS Reality Check

My primary background is iOS development, where I've spent years building native Swift applications. AI tools in this domain remain frustratingly limited. The Apple ecosystem's complexity—from UIKit intricacies to SwiftUI's evolving patterns, Xcode's quirks, and platform-specific constraints—proves challenging for current AI models. While AI can generate basic Swift code, it struggles with the nuanced understanding required for production iOS development. It often writes outdated code, particularly when it comes to Swift 6 and concurrency. It can quickly iterate itself into a downward spirall of overcomplicated code which can be a complete mess. The feedback loops are slow, and the tools lack deep integration with Xcode workflows that iOS engineers rely on daily.

I've experimented with MCP implementations for Xcode, but the integration remains rudimentary compared to what's needed for serious iOS work. Claude often calls the tool wrong, tries a few times, and only then gets it right. This can be somewhat mitigated by having a well written CLAUDE.md file but it's not perfect.

## Web and Backend: A Different Story

The contrast with web development is stark. When working outside my primary domain—web frontend and backend development—AI tools prove remarkably effective. I recently built a complete [web version](https:///www.swimtemps.com) of my iOS app Swim using pretty much exclusively Claude Code. What would have taken weeks of research, boilerplate generation, and stack decisions was completed in a fraction of the time.

For Rust backend development, similar results emerge. The AI handles dependency management, error handling patterns, and API design with competence that approaches what I'd expect from a junior engineer familiar with the stack. However, due to Rust's intricacies it will still get things wrong. Armin Ronacher [recently wrote](https://lucumr.pocoo.org/2025/6/12/agentic-coding/) about LLMs being quite good at Go so I will give that a try next (and coincidentally, we chose Go as the primary backend stack at Specto for similar reasons why it works well with LLMs).

## Interaction Paradigm

The most significant shift isn't the code generation itself, but the interaction model. Rather than wrestling with command-line syntax or documentation, I can communicate with AI as I would with an engineer on my team. This conversational interface for system administration, project setup, and routine development tasks represents a meaningful productivity improvement. I generally run up to 3 terminal tabs with Claude in each one, either working on a different codebase/repository or even a different checkout of the same one.

I've been experimenting with bypassing permission prompts to increase iteration speed—a calculated risk that requires robust backup systems but dramatically improves the development flow. The AI maintains context across tasks, enabling complex multi-step operations that would typically require careful orchestration.

Claude Code is much easier to interact with in this new paradigm than Cursor. It lives in the terminal, focuses purely on the agentic aspect and doesn't even provide an editor. Cursor, as a vscode fork, is much more complicated and has a ton of options. Now ironically this post is being written in Cursor with a markdown preview plugin so there's still place for editors but I think Claude Code is on a better path.

## MCP and Automation

Beyond code generation, I've tested MCP implementations for browser automation with Puppeteer. The ability to describe user interface interactions in natural language and have them translated to reliable automation scripts opens interesting possibilities for testing and data collection workflows. This works pretty well most of the time but again, a well written CLAUDE.md file works wonders. As mentioned above, this works to some extent with Xcode as well but it's much more cumbersome. It will be interesting to follow the progress there. I've also tried using an MCP server for Railway (which is where I host most of my projects) and Vercel. For both I found it much easier to just get Claude to use the CLI tools for which I provided basic instructions in the respective CLAUDE.md files. The contents are basically the outputs of `vercel --help` and `railway --help`, condensed and with instructions to call the tool and command with `--help` to learn more when something doesn't work. Work wonders.

## Assessment

The current state suggests AI development tools are most valuable when working outside your core expertise. For an iOS engineer like myself, they excel at web development tasks I'd otherwise need to research extensively. The tools serve as force multipliers for exploring adjacent technologies rather than replacing deep domain knowledge (I'm convinced this will change super quickly though).

The interaction paradigm represents the more fundamental shift. We're moving toward intent-driven development where you describe desired outcomes rather than specifying implementation details. This abstraction works well for certain problem domains while remaining limited in others.

For now, AI development tools are powerful assistants rather than replacements, with effectiveness varying significantly by technology stack and problem complexity. It is starting to feel like you can start a company with less employees, particularly around building a website, blog, doing marketing, etc. The non-coding aspect is something I will explore more of next (I just connected Claude to my gmail inbox...).

**Disclaimer:** This post was written by me but proofread and formatted by Claude. Yet another way Claude helps me lately, especially as a non-native English speaker!


Jernej Štrasner

Blog

Learning How Chunking, Embeddings, and Vector Databases Actually Work

AI Development Tools: A Pragmatic Assessment