Introduction

smart_toyOpenAI Integration

Use the generated JSON Schema with OpenAI's Function Calling for structured product queries.

// Using JSON Schema with OpenAI Function Calling
const response = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [{ role: "user", content: "Find red shoes under $50" }],
  functions: [{
    name: "search_products",
    parameters: jsonSchema // Use generated JSON Schema
  }]
});

storageVector Database Import

Import NDJSON output into Pinecone or other vector databases for semantic search.

// Importing NDJSON to Pinecone
const fs = require('fs');
const ndjsonLines = fs.readFileSync('products.ndjson', 'utf-8').split('\n');

for (const line of ndjsonLines) {
  if (line.trim()) {
    const product = JSON.parse(line);
    await pineconeIndex.upsert({
      vectors: [{
        id: product.id,
        metadata: product,
        values: await embed(product.triples.join(' '))
      }]
    });
  }
}

checklistData Quality Checklist

• check_circleEnsure CSV is UTF-8 encoded for proper character handling
• check_circleVerify required columns (Handle, Title*, Vendor, Tags, Collections)
• check_circleCheck for duplicate SKUs to avoid data conflicts
• check_circleValidate image URLs are accessible and properly formatted
• check_circleRemove any unnecessary HTML from product descriptions

datasetLarge Dataset Strategy

• tips_and_updatesSplit files larger than 10,000 products for better performance
• tips_and_updatesUse batch mode for multiple files to merge results efficiently
• tips_and_updatesMonitor browser memory usage when processing large datasets
• tips_and_updatesConsider using NDJSON for streaming large datasets to databases