Files
jarvis-at-skic f39c2a1e5f
Build and Deploy Documentation / build (push) Failing after 1m24s
feat: Initial Docusaurus setup with Stock Market Professional docs
- Set up Docusaurus with pnpm and TypeScript
- Structured docs for multi-application portfolio
- Added Stock Market Professional documentation:
  - Architecture & system design
  - Data monitoring module
  - Technical analysis engine
  - Signal generation with constraints (max 3/day, 30min hold)
- Added platform architecture overview
- Added getting started guide
- Added Gitea Actions CI/CD pipeline (build + artifact)
2026-07-16 19:47:40 +00:00

2.3 KiB

sidebar_position
sidebar_position
2

Data Monitoring

Overview

The data monitoring module is responsible for acquiring, normalizing, and storing market data that feeds into the analysis engine.

Requirements

  • Latency: Near real-time (< 1 minute delay)
  • Coverage: Forex major pairs + selected stocks/indices
  • Granularity: 1-minute candles, tick data where available
  • History: Minimum 30 days rolling window for backtesting
  • Reliability: Automatic reconnection, data gap detection

Watched Instruments

Forex Pairs (Initial)

Pair Session Notes
EUR/USD London + NY Most liquid, tight spreads
GBP/USD London + NY High volatility windows
USD/JPY Tokyo + London Trend-following opportunities
AUD/USD Sydney + London Commodity correlation

Indices (Future)

Index Session Notes
S&P 500 (SPX) NY Broad market sentiment
NASDAQ 100 (NDX) NY Tech sector proxy

Data Pipeline

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  API Source  │────▶│  Normalizer  │────▶│   Storage    │
│              │     │              │     │              │
│ • REST poll  │     │ • OHLCV fmt  │     │ • SQLite/TS  │
│ • WebSocket  │     │ • Timestamps │     │ • Partitioned│
│ • Fallback   │     │ • Validation │     │ • Indexed    │
└──────────────┘     └──────────────┘     └──────────────┘

Data Schema

CREATE TABLE candles (
    id INTEGER PRIMARY KEY,
    symbol TEXT NOT NULL,
    timeframe TEXT NOT NULL,  -- '1m', '5m', '15m', '1h'
    timestamp DATETIME NOT NULL,
    open REAL NOT NULL,
    high REAL NOT NULL,
    low REAL NOT NULL,
    close REAL NOT NULL,
    volume REAL,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE(symbol, timeframe, timestamp)
);

CREATE INDEX idx_candles_symbol_time ON candles(symbol, timeframe, timestamp DESC);

Health Checks

  • Data freshness monitoring (alert if > 5 min stale)
  • Gap detection and backfill
  • Source failover (primary → secondary API)
  • Rate limit tracking per API key