Small Embedding Model (384 Dimensions)

Our small embedding model (BGE Small) provides efficient vector representations in 384 dimensions, perfect for applications where speed and resource efficiency are priorities while maintaining good semantic understanding.

Model Overview

BGE Small v1.5

• Dimensions: 384
• Provider: BAAI
• License: Apache 2.0
• Max Tokens: 8192
• Model Card: BAAI/bge-small-en-v1.5

Configuration & Setup

OpenAI SDK Setup

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: 'your-neuredge-key',
  baseURL: 'https://api.neuredge.dev/v1/'
});

Native SDK Setup

import { Neuredge } from '@neuredge/sdk';

const neuredge = new Neuredge({
  apiKey: 'your-api-key'
});

Real-World Applications & Examples

1. Efficient Search Systems

Basic Semantic Search

// Generate embeddings for documents
const documentEmbeddings = await openai.embeddings.create({
  model: '@cf/baai/bge-small-en-v1.5',
  input: documents.map(doc => doc.content)
});

// Generate embedding for search query
const queryEmbedding = await openai.embeddings.create({
  model: '@cf/baai/bge-small-en-v1.5',
  input: searchQuery
});

// Example using pgvector
const results = await db.query(`
  SELECT id, content, 
    embedding <=> $1 as distance
  FROM documents
  ORDER BY distance
  LIMIT 5
`, [queryEmbedding.data[0].embedding]);

Use Cases:

• Document retrieval
• Content search
• FAQ matching
• Knowledge base search
• Resource discovery

2. Product Recommendations

E-commerce Product Matching

// Generate embeddings for product descriptions
async function getProductEmbeddings(products) {
  const response = await openai.embeddings.create({
    model: '@cf/baai/bge-small-en-v1.5',
    input: products.map(p => 
      \`${p.name} ${p.description} ${p.category}\`
    )
  });
  return response.data.map(d => d.embedding);
}

// Find similar products
async function findSimilarProducts(productId, embeddings, k = 5) {
  const productEmbedding = embeddings[productId];
  const similarities = embeddings.map((emb, idx) => ({
    id: idx,
    similarity: cosineSimilarity(productEmbedding, emb)
  }));
  
  return similarities
    .sort((a, b) => b.similarity - a.similarity)
    .slice(1, k + 1); // Exclude the product itself
}

Use Cases:

• Product recommendations
• "Similar items" features
• Cross-selling suggestions
• Category organization
• Product discovery

3. Content Organization

Document Clustering

import { kmeans } from 'ml-kmeans';

// Generate embeddings for documents
const response = await openai.embeddings.create({
  model: '@cf/baai/bge-small-en-v1.5',
  input: documents.map(doc => doc.content)
});

const embeddings = response.data.map(d => d.embedding);

// Perform k-means clustering
const { clusters } = kmeans(embeddings, {
  k: 5,
  initialization: 'kmeans++',
  iterations: 100
});

Use Cases:

• Content categorization
• Topic modeling
• Document organization
• Content discovery
• Trend analysis

4. Duplicate Detection

Content Deduplication System

async function findDuplicates(texts, threshold = 0.95) {
  const response = await openai.embeddings.create({
    model: '@cf/baai/bge-small-en-v1.5',
    input: texts
  });

  const embeddings = response.data.map(d => d.embedding);
  const duplicates = [];

  for (let i = 0; i < embeddings.length; i++) {
    for (let j = i + 1; j < embeddings.length; j++) {
      const similarity = cosineSimilarity(
        embeddings[i],
        embeddings[j]
      );
      if (similarity > threshold) {
        duplicates.push([i, j]);
      }
    }
  }

  return duplicates;
}

Use Cases:

• Content deduplication
• Plagiarism detection
• Similar content grouping
• Data cleaning
• Quality control

Integration Examples

Express.js with Vector Database

import express from 'express';
import OpenAI from 'openai';
import { Pool } from 'pg';
import pgvector from 'pgvector/pg';

const app = express();
const pool = new Pool();
await pool.query('CREATE EXTENSION IF NOT EXISTS vector');

const openai = new OpenAI({
  apiKey: process.env.NEUREDGE_API_KEY,
  baseURL: 'https://api.neuredge.dev/v1/'
});

app.post('/search', async (req, res) => {
  try {
    const { query } = req.body;
    
    // Generate query embedding
    const embedding = await openai.embeddings.create({
      model: '@cf/baai/bge-small-en-v1.5',
      input: query
    });

    // Search vector database
    const results = await pool.query(`
      SELECT content, 
        1 - (embedding <=> $1) as similarity
      FROM documents
      WHERE 1 - (embedding <=> $1) > 0.7
      ORDER BY similarity DESC
      LIMIT 5
    `, [embedding.data[0].embedding]);

    res.json(results.rows);
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

Next.js Search Implementation

// pages/api/search.js
import OpenAI from 'openai';
import { createClient } from '@supabase/supabase-js';

const openai = new OpenAI({
  apiKey: process.env.NEUREDGE_API_KEY,
  baseURL: 'https://api.neuredge.dev/v1/'
});

const supabase = createClient(
  process.env.SUPABASE_URL,
  process.env.SUPABASE_KEY
);

export default async function handler(req, res) {
  if (req.method !== 'POST') {
    return res.status(405).json({ message: 'Method not allowed' });
  }

  try {
    const { query } = req.body;

    const embedding = await openai.embeddings.create({
      model: '@cf/baai/bge-small-en-v1.5',
      input: query
    });

    const { data, error } = await supabase.rpc('match_documents', {
      query_embedding: embedding.data[0].embedding,
      match_threshold: 0.7,
      match_count: 5
    });

    if (error) throw error;
    res.status(200).json(data);
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
}

Best Practices

1. Data Preparation

   • Clean input text
   • Handle special characters
   • Normalize content length
   • Remove irrelevant content

2. Vector Operations

   • Use efficient similarity metrics
   • Implement proper indexing
   • Batch similar operations
   • Cache common embeddings

3. System Design

   • Choose appropriate vector DB
   • Plan for scaling
   • Monitor performance
   • Implement error handling

Token Management

Plan	Monthly Token Quota
Free Tier	300K tokens
$29 Plan	3M tokens
$49 Plan	4.5M tokens

When to Use

✅ Ideal For:

• Resource-constrained environments
• High-throughput applications
• Mobile/edge deployment
• Quick prototyping
• Basic similarity search

❌ Consider Alternatives When:

• Maximum accuracy needed
• Complex semantic tasks
• Deep understanding required
• Research applications

Getting Started

To begin using BGE Small embeddings:

• Quick Start Guide
• JavaScript SDK
• Python SDK