Tencent Cloud DataBuddy covers the entire big data lifecycle. By module, it can be divided into eight core capabilities, which correspond to the eight top-level directories in the product documentation.
Data Access
Intelligent Data Access: It evolves from "manual configuration" to "conversational creation", and from "passive Ops" to "AI-driven proactive diagnosis".
40+ Data Sources: Relational databases, data warehouses, lakehouses, NoSQL, KV, message queues, streams, files, COS, APIs, and multimodal data.
Five Major Access Solutions:
Offline Synchronization (Batch, Incremental, Scheduled, Four Major Link Types)
Real-time Synchronization (CDC Incremental, Whole Database with Multiple Tables, Sharded Databases and Tables)
File to Table (CSV, Excel, JSON, Automatic Table Creation)
File to Volume (Unstructured Data, Images, Videos, Documents)
Batch to Volume (FTP, S3, Preserving Directory Structure)
DataBuddy Agent Intelligent Assistance: It enables task creation via natural language, AI-powered intelligent Q&A, and AI-driven root cause diagnosis for task exceptions.
Data Engineering
Unified Data Development Workbench, Orchestration, CI/CD, and Ops: DIOPS forms a closed loop through its four major modules, covering the entire data development lifecycle.
Studio (Development)
Notebook: It features multi-language cells (Python, SQL, Scala, Markdown) and magic command switching.
SQL IDE: It provides unified Spark SQL dialect (Kyuubi), metadata-aware auto-completion, and parameterized queries.
Python IDE: It supports cross-file imports, Debug mode, and full Python 3 syntax.
Intelligent Auto-completion: Catalog, Schema, Table, Column
AI Assistant DataBuddy: It generates SQL from natural language, corrects code errors, provides automatic comments, and continues code writing.
Native Git Integration: commit, push, pull, branch management
Workflow (Orchestration and Scheduling)
Visual DAG: It enables drag-and-drop task addition, dependency creation via connections, trigger configuration, and immediate effect upon saving.
9 Task Types: Notebook, SQL, Python, Ray, Offline Ingestion, IF-ELSE, ForEach, Nested, Quality Monitoring
12 Run Conditions: All Success, All Failure, At Least One Success, At Least One Completion, and so on
Scheduling Triggers: Regular intervals (day, hour, minute, week, month, year), Cron expressions, time zones.
Continuous Run: It is automatically triggered 60 seconds after the previous terminal state, making it suitable for near-real-time scenarios.
Parameter Management: two-level parameters, inter-task passing, built-in parameters
CI/CD (Release)
Configuration as Code: A Bundle publishing system based on YAML and CLI.
Multi-Environment: dev, staging, prod automated deployment
Command Set: bundle init, validate, deploy, run, summary, destroy
OPS (Operations Monitoring)
Four-Layer Run Monitoring Views: workflow run list, run records, run details, task run details
Multi-Channel Alarms: Email, Webhook, Teams, WeCom, Slack
AI Intelligent Diagnosis: instance information, root cause analysis, impact scope, remediation suggestions
Data Science
OneOps: An Integrated Platform for Enterprise AI Engineering: DataOps, MLOps, AIOps.
Five Core Modules
|
Studio | Integrated development environment (IDE) with Notebook and built-in Copilot support |
Feature Management Feature Store | Feature CRUD, built-in operators, offline-online consistency, cross-project reuse |
Model Experiment Experiment | MLflow tracking, AutoML, large model experimentation, hyperparameter search, experiment reproduction |
Model Management Registry | Model registration, version management, model lineage, one-click promotion of champion models |
Model Serving Serving | Online serving, batch prediction, multi-version A/B testing, elastic scaling, automatic start-stop |
Data analysis
From "Waiting for Scheduling" to "Conversation as Insight": Designed for analysts and business personnel.
Intelligent Data Query: It enables data retrieval in seconds by leveraging the semantic layer through natural language queries.
AI-Generated Analysis Reports: It generates comprehensive reports with one click and supports proactive, scheduled delivery.
Proactive Intelligent Attribution: It performs multi-dimensional attribution using AI and outputs actionable recommendations.
AI-Generated Dashboards: It automatically plans charts and generates dashboards based on natural language descriptions.
Data Exploration: interactive data exploration, direct SQL query
Analysis Space: It enables the creation and management of analysis spaces and supports their embedding.
Data Governance
From "Manual Governance" to "AI-Powered Intelligent Guarding": DataBuddy's Four Core Capabilities.
Unified Semantic Layer Unity Semantics
Ontology modeling standardizes entities, metrics, dimensions, and business logic.
Eliminate metric ambiguity and avoid SQL hallucinations.
NL2SQL Accuracy: Without a semantic layer, accuracy is between 60% and 70%; with a complete semantic layer, accuracy is 90%+.
Data Quality
Rule Engine: It provides one-stop rule definition in YAML and supports multiple rule types (null rate, uniqueness, value range, fluctuation rate, output timeliness, custom SQL).
Scheduling Execution: It can be embedded into ETL pipelines, configured for pre-execution blocking, and integrated with workflows.
Rule-Level Precise Alarms: It pushes alarms through multiple channels (WeCom, Email, Webhook).
Anomaly Detection: It can be enabled with one click at the Schema level, requiring zero configuration and featuring self-learning capabilities.
Data Profiling: It supports three modes (Snapshot, Time Series, Inference) and drift metrics (KS, PSI, Wasserstein, Chi-Square, JS Divergence).
Multimodal Data Catalog TCCatalog
Table Catalog: Tables, Views, Functions
Volume Catalog: Logs, Images, PDFs
Model Catalog: ML Models, Version Management
External Catalog: A mirror of external data sources, enabling querying without ETL.
End-to-End Lineage: It automatically collects SQL and integration tasks, and provides dual views in both graph and list modes.
Agent Security Protection
Bidirectional Agent Guardrails: It performs real-time detection of prompt injection and blocks attack chains.
OBO Permission Strong Binding: It reuses the permissions from the underlying data platform based on the real user identity.
High-Risk SQL Interception and Intent Verification: It performs high-risk assessment before invocation and execution.
"Three-Factor" Precision Interception: It triggers manual confirmation when untrusted input, state changes, and sensitive data are all matched simultaneously.
Cover all risks in the OWASP LLM Top 10
Data Applications
Agents, MCP, and Model List Build the Application Ecosystem.
Agent Overview: It provides three built-in Agents and custom Agent capabilities.
MCP: It provides native support for the Model Context Protocol, enabling both humans and Agents to use the same invocation chain.
Model List: Multi-Model Integration and Endpoint Management
Buddy (Agent Capabilities)
Buddy Agent is the core differentiating capability of this product. For details, see the Buddy top-level directory.
Buddy Overview: Product Positioning and Two Major Scenario Modes (Data Analysis, Data Development)
Buddy - Data Analysis Scenario: Analysis Space Management, Complex Data Analysis, AI Dashboard Building, and Analysis Space Embedding
Data Engineering Agent: From "Manual Setup" to "Agent End-to-End Delivery"
Data Governance Agent: From "Manual Governance" to "AI-Powered Intelligent Guardianship"
Buddy Data Security: Agent Security Protection System
Buddy Agent Basic Capabilities: Knowledge Base, Data Review, @ Mention, Ask Mode, File Upload, Artifacts and Artifact Changes
Platform management
Workspace Management: Workspace Creation, Member and Permission Management, Workspace Settings (Notification Channels, Git Connection, Personal Preferences)
Data Source Management: Global Data Source Configuration, Adding/Modifying Data Sources, Connectivity Testing
Compute Resource Management: Compute Resource Overview, Creation, Network Configuration, Usage Monitoring