Skip to content

New article: Joint Embedding Spaces for Multimodal Search #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DigitalTsotsi wants to merge 1 commit into
base: main
Choose a base branch
from
Open

New article: Joint Embedding Spaces for Multimodal Search #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions articles/20260808-multimodal-embeddings.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Joint Embedding Spaces for Multimodal Search &ndash; VModal Blog</title>
<link rel="stylesheet" href="../style.css" />
</head>
<body>
<header>
<a href="../index.html">V-Modal AI Blog</a>
</header>

<main>
<a class="back-link" href="../index.html">&larr; Back to all articles</a>

<div class="article-body">
<h1>Joint Embedding Spaces for Multimodal Search</h1>
<p style="color:#888; font-size:0.9rem; margin-bottom:1.5rem;">August 2026 &middot; AI &middot; Search</p>

<p>Multimodal search rests on a simple idea: project every modality into one shared vector space where similar things land close together, regardless of whether they started as text, an image, or a sound. Once everything is a vector, retrieval becomes a nearest-neighbour lookup.</p>

<h2>Why a shared space</h2>

<p>A shared space lets a text query match an image, or an image match a sound, without any hand-built mapping between them. The model learns the alignment from data, so a photo of a red bicycle and the phrase "red bicycle" end up as neighbours.</p>

<h2>How the alignment is learned</h2>

<p>Contrastive training does the work. The model is shown matched and mismatched pairs and learns to pull matches together while pushing mismatches apart:</p>

<ul>
<li>Supervision comes from naturally occurring pairs, such as images and their captions.</li>
<li>No manual labelling is needed, so the approach scales to hundreds of millions of pairs.</li>
<li>The same recipe extends to audio, depth, and other modalities.</li>
</ul>

<h2>Turning embeddings into search</h2>

<p>At query time the system encodes the query with the same model and looks up the closest vectors in an index. The quality of the embedding decides the ceiling on relevance; the index decides how fast you reach it.</p>
</div>
</main>

<footer>
&copy; 2026 VModal Blog
</footer>
</body>
</html>
5 changes: 5 additions & 0 deletions index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,11 @@ <h2>🚀 Moving Forward</h2>

<h2>List of Blog Articles</h2>
<ul class="article-list">
<li>
<a href="articles/20260808-multimodal-embeddings.html">Joint Embedding Spaces for Multimodal Search</a>
<div class="summary">How shared vector spaces let a single query reach across text, images, and audio.</div>
<div class="meta">August 2026 &middot; AI &middot; Search</div>
</li>
<li>
<a href="articles/202509-video-preview-tool.html">How to Build a Video Preview Tool for Mobile</a>
<div class="summary">A practical guide for mobile developers on decoding, frame sampling, temporal segmentation, and network optimization for on-device video analysis.</div>
Expand Down