Skip to main content

How to specify the time interval for historical data in AutoConfig?

· 7 min read

Gunbot's AutoConfig feature uses historical ticker data for in-depth market analysis by its filters. Alongside setting the amount of historical data (via the history parameter), it's crucial to define the time interval between these data points. This is achieved using the historyInterval parameter in your AutoConfig job configuration, allowing you to control the granularity and time span of your historical dataset.

Use the AutoConfig wizard

Nearly every option that follows can be set without editing files by hand.
Click the ⋮ (three-dots) menu ➜ AutoConfig, step through the wizard, and press Save; it will write a correct autoconfig.json for you.

The Significance of historyInterval​

AutoConfig maintains two types of ticker data:

  1. Snapshots: These are frequent, recent market data captures, configured by the snapshots parameter. They provide a detailed view of the very recent past.
  2. Historical Data: These are less frequent data points, drawn from the oldest available snapshots, and stored for longer-term analysis. The history parameter defines how many such points are kept.

The historyInterval parameter bridges these two data types. It specifies how much time should elapse before AutoConfig considers moving an old snapshot into the historical data pool. The value of historyInterval is typically interpreted in minutes. Internally, Gunbot often converts this to seconds (e.g., historyInterval * 60) for comparison with timestamps.

When the oldest snapshot in the snapshots array becomes older than the historyInterval relative to the most recent historical data point (or since the job started, if no history exists yet), it's eligible to be added to the history array.

Configuring historyInterval​

The historyInterval is set as a numerical value within your AutoConfig job's JSON definition. This number represents the interval in minutes.

Here’s how it looks in an example AutoConfig job:

{
  "longTermTrendJob": {
    "enabled": true,
    "schedule": "*/5 * * * *", // Job runs every 5 minutes
    "type": "addPairs",
    "snapshots": 120, // Store 120 recent 5-minute snapshots (10 hours of recent data)
    "history": 200,   // Store 200 historical data points
    "historyInterval": 30, // User setting: Interval in minutes for historical data points
    "pairs": {
      "exchange": "bitget",
      "include": "USDT-BTC,USDT-ETH"
      // ... other pair settings
    },
    "filters": {
      "customHistoryCheck": {
        "filterType": "custom",
        "script": " /* JavaScript using this.history for analysis */ "
      }
    }
    // ... other job settings
  }
}

In the "longTermTrendJob" example:

  • historyInterval: 30 means that a new historical data point will be logged approximately every 30 minutes.
  • With history: 200, this job would store historical data covering a period of 200 * 30 minutes = 6000 minutes, which is 100 hours (or just over 4 days).

The actual timing of when a snapshot transitions to history depends on the job's schedule and the timestamps of the collected snapshots. If a job runs every 5 minutes, the oldest snapshot from the snapshots pool will be considered for promotion to the history pool if it's suitably old according to historyInterval relative to the latest history entry.

How historyInterval Affects Your Strategy​

The choice of historyInterval significantly influences the nature of the historical data available to your AutoConfig filters:

  1. Data Granularity:

    • A smaller historyInterval (e.g., 5 or 10 minutes) means historical data points are closer together in time. This provides a more detailed, granular view of the past but, for a fixed history count, covers a shorter total time span.
    • A larger historyInterval (e.g., 60 or 240 minutes) means historical data points are further apart. This offers a view over a longer total duration but with less detail about fluctuations between these points.

  2. Total Historical Span: The total time period covered by your historical data is the product of history * historyInterval.

    • history: 100, historyInterval: 15 minutes -> 100 * 15 = 1500 minutes (25 hours) of history.
    • history: 100, historyInterval: 60 minutes -> 100 * 60 = 6000 minutes (100 hours) of history.

  3. Filter Sensitivity: Filters that analyze trends or compare values across historical points (e.g., minSlopePctIntervalHistory) will behave differently based on historyInterval. A slope calculated over hourly intervals will reflect broader trends than a slope calculated over 5-minute intervals.

  4. Resource Usage: While historyInterval itself doesn't directly increase data storage (that's history's role), a very small historyInterval might lead to more frequent processing of historical data if other logic is tied to updates in the history array. However, this is generally a minor factor compared to the history count.

Selecting the Right historyInterval​

The ideal historyInterval depends on the timeframes relevant to your trading strategy:

  • For analyzing intra-day trends (few hours to a day): Intervals like 15, 30, or 60 minutes are common.
    • Example: history: 96, historyInterval: 15 (24 hours of data, 96 points).
  • For analyzing multi-day or weekly trends: Longer intervals like 120, 240, or even 1440 minutes (1 day) might be used.
    • Example: history: 168, historyInterval: 60 (7 days of data, 168 hourly points).
  • Balancing Detail and Span: If you need to see fine-grained changes but also want a long history, you'll need a larger history value to compensate for a smaller historyInterval. Conversely, if only broad strokes of long-term history matter, a larger historyInterval allows a smaller history count to cover the desired period.

Practical Tip: Ensure that your historyInterval is greater than your job's schedule interval. If your job runs every 5 minutes, a historyInterval of 1 or 2 minutes would not be logical, as new snapshots are not generated that frequently by that specific job. AutoConfig typically promotes the oldest snapshot from the snapshots array when its timestamp is appropriately older than the last history entry, based on historyInterval.

Using historyInterval with historySource in Filters​

Many built-in AutoConfig filters that operate on historical data include a historySource parameter. This parameter usually specifies an index in the this.history[jobName] array. The historyInterval gives context to what each index represents in terms of time.

For instance, if historyInterval: 60 (minutes):

  • historySource: 0 refers to the oldest data point, roughly history * 60 minutes ago.
  • historySource: 1 refers to the next data point, roughly (history - 1) * 60 minutes ago.

Understanding this relationship is key to correctly configuring filters like minPricePctChangeIntervalHistory.

// Conceptual JavaScript usage in a custom filter
// this.history[jobName] is an array of historical snapshots.
// The time difference between historicalData[i] and historicalData[i+1] is
// determined by the 'historyInterval' (user setting).

const jobName = 'myJob'; // Placeholder
const historicalData = this.history[jobName];
const configuredIntervalMinutes = this.config.AutoConfig[jobName].historyInterval;

if (historicalData && historicalData.length >= 2) {
  const latestHistoryPoint = historicalData[historicalData.length - 1];
  const secondLatestHistoryPoint = historicalData[historicalData.length - 2];

  // Timestamps are usually in seconds Unix format
  const timeDiffSeconds = latestHistoryPoint.timestamp - secondLatestHistoryPoint.timestamp;
  const timeDiffMinutes = timeDiffSeconds / 60;

  // timeDiffMinutes should be approximately equal to configuredIntervalMinutes
  // (allowing for minor variations due to job scheduling)

  // console.log(`Expected interval: ${configuredIntervalMinutes} min, Actual interval: ${timeDiffMinutes.toFixed(2)} min`);
}

This example illustrates how historyInterval defines the temporal spacing of data points within the this.history array, which you might use in custom JavaScript filters.

By thoughtfully choosing the historyInterval, you fine-tune the lens through which AutoConfig views market history, ensuring that the data it analyzes is relevant and appropriately scaled for your trading strategy's time horizon.