Data Pipeline data freshness

Understand data availability timelines for Data Pipeline exports.

Data Pipeline shares your Stripe data to your destination on a recurring schedule. The time it takes for data to arrive varies by dataset.

Note

Data delivery varies from customer to customer and is largely determined by the data sizes. After sign-up, you can see your actual delivery time in your warehouse.

The following terms describe Data Pipeline timing:

• Cadence : How frequently Stripe shares data to your destination. For example, a cadence of 3 hours means data is shared at 00:00, 03:00, 06:00 UTC, and so on.
• Freshness P50 (hours) : 50% of the time, data is available within this duration from event creation.
• Freshness P90 (hours) : 90% of the time, data is available within this duration from event creation.
• Max delay (hours) : The maximum expected time from event creation to data availability.

For example, with a 3-hour cadence, the 03:00 UTC batch includes a record created at 00:01 UTC. With a typical delivery time of 3 hours after the batch closes, the data is available by 06:00 UTC—a P50 freshness of 6 hours from event creation.

Understanding your pipeline’s data freshness

In your Stripe Dashboard, you can view data landing and “data as of” times by dataset. Here are the definitions of all the timestamps:

• Data landed : The last time the data landed in your destination.
• Data as of : All API-backed data until this timestamp is included in the exported snapshot. This varies for derived datasets.
• Next sync start time : The next time your data begins loading.

Query data load times with Data Pipeline

Stripe provides the data_load_times table as part of your data share. In your warehouse, you can see when the data in each table is updated by viewing the date in the the related setting column.

You can use the above values in your queries to filter data based on when it’s most recently updated in each table. Use the following query to get data about your balance transactions from the last week:

select
 created,
 id,
 amount,
 currency,
 source_id
from
 balance_transactions,
 data_load_times
where
 data_load_times.table_name = 'balance_transactions'
 and datediff(day, created, data_load_times.loaded) <= 7
order by
 created desc

Dataset freshness