Scenarios
Download
포커스 모드
폰트 크기
In traditional offline data warehouses, the flow of data from ODS to DWD, DWS, and finally ADS is typically completed via T+1 batch processing, which struggles to meet business requirements for real-time performance. Setats inherently supports a real-time layered data warehouse architecture: a complete Changelog is automatically generated for each layer's Setats table upon data updates. Downstream layers then perform incremental computations by subscribing to the Changelog from upstream layers. This process is linked layer by layer, forming a real-time incremental data pipeline from the data source to the final aggregated result.
For example, after raw data from the ODS layer is written to Setats, a Flink job subscribes to its Changelog for cleansing and correlation, and writes the results to a Setats table in the DWD layer. Changes in the DWD layer then trigger downstream Flink jobs to complete metric aggregation and write the results to the DWS layer. The entire process achieves end-to-end latency at the second level. Data changes at each layer are propagated downstream in real time, eliminating the need to wait for batch processing scheduling.
Meanwhile, each layer of Setats tables supports both Batch and OLAP queries. Users can directly perform interactive queries and multi-dimensional analysis on data at any layer using engines such as Doris, StarRocks, and Spark, significantly enhancing the flexibility of data exploration and the efficiency of problem troubleshooting.
With this architecture, users can use a single Setats storage to simultaneously meet the dual demands of real-time incremental streaming and data analysis. This eliminates the need to move data between message queues, data lakes, and OLAP engines, significantly simplifying the data warehouse architecture and reducing Ops costs.
The Tencent Cloud Stream-Lake Engine Setats solution can be widely applied across multiple industries and scenarios, including but not limited to sectors such as mobility, gaming, education, and e-commerce.
Mobility: Vehicles continuously report a large volume of signal data, such as location, speed, fuel/electricity level, and tire pressure, through in-vehicle terminals. This data is characterized by high write frequency, numerous fields, and frequent updates. Setats can directly handle the real-time ingestion of these in-vehicle signals. Using the vehicle ID as the primary key and leveraging Partial Update, it enables independent updates for each signal field, eliminating the need to report all fields every time. Setats supports efficient writes to wide tables with thousands of columns and ensures data visibility within seconds. Dispatch systems can query the latest status of any vehicle in real time for dynamic route planning and vehicle allocation. Historical signal data is persisted to remote columnar storage for use in offline scenarios such as trajectory backtracking, fault analysis, and operational reporting.
Gaming: User acquisition is a core growth strategy, requiring real-time tracking of advertising campaign performance and user conversion funnels. Setats supports the real-time ingestion of user acquisition event data from various channels. It then uses Changelog to drive Flink jobs for real-time attribution calculation and metric aggregation. Ops personnel can obtain the performance of campaigns across different channels, creatives, and regions within seconds using Doris or StarRocks. This enables them to quickly identify high-ROI channels and promptly shut down inefficient campaigns, significantly improving user acquisition efficiency and budget utilization.
Education: Setats can help educational institutions track students' learning progress and behavioral patterns in real time. Course platforms write students' learning events, such as course viewing progress, answer records, and interaction behaviors, to Setats in real time. Teachers and administrative staff can then promptly understand each student's learning status based on data visible within seconds, provide personalized teaching suggestions, quickly identify students with learning difficulties, and offer targeted support.
E-commerce: Setats can help merchants achieve precise user Profile Analytics. By writing users' behavioral data, such as browsing, searching, adding to cart, and placing orders, to Setats in real time, and leveraging the Partial Update capability to continuously update various dimensions of user profiles, the marketing system can obtain the latest user preferences in real time. This enables it to promptly adjust recommendation algorithms and marketing policies, quickly respond to market changes, and optimize promotional activities, thereby increasing conversion rates and customer satisfaction. Furthermore, Setats supports efficient writes and queries for large volumes of product and order data, ensuring that merchants can obtain sales data in real time and make timely inventory and marketing adjustments.
피드백