Agent-ready OpenAPI document · curated by JenticAnalyticsData PipelinesAWS Signature v4 (HMAC)19 EndpointsREST
Know of an official OpenAPI document? Contribute it →
For Agents
Manage AWS Data Pipeline definitions, activate and deactivate pipelines, and coordinate task runners.
Use for: I need to activate an existing data pipeline, I want to validate a new pipeline definition before activation, List all pipelines in this account, Describe the status of the objects in a running pipeline
Not supported: Does not handle modern serverless orchestration or Spark-based ETL - use AWS Data Pipeline only for managing existing legacy pipelines; use Step Functions or Glue for new work.
Jentic publishes the only available OpenAPI specification for AWS Data Pipeline, keeping it validated and agent-ready. AWS Data Pipeline configures and runs data-driven workflows that move and transform data between AWS services and on-premises sources. The API manages pipeline definitions, activations, task runners, and validation. AWS announced Data Pipeline as legacy in 2023 and recommends Step Functions, Glue Workflows, or MWAA for new work, but the API is still available for existing pipelines.
Install Jentic One Beta
Connect the AWS Data Pipeline to your agent
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the AWS Data Pipeline, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
1
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Famazonaws.com%2Faws-datapipeline" | sh2
Step 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Famazonaws.com%2Faws-datapipeline" | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
Capabilities
What an agent can do with AWS Data Pipeline API.
Create pipelines and put their definitions with PutPipelineDefinition
Activate, deactivate, and delete pipelines to control execution
Validate a pipeline definition before activation
Poll for and report on tasks executed by task runners
List pipelines and describe the status of objects within them
Add tags and query objects with arbitrary expressions
Use Cases
Patterns agents use AWS Data Pipeline API for, with concrete tasks.
★ Maintaining a Legacy ETL Pipeline
Many organizations still operate AWS Data Pipeline workflows that move data between S3, RDS, Redshift, and on-premises sources on a schedule. The API allows operators to put updated definitions, activate or deactivate runs, and inspect status without using the deprecated console.
Call PutPipelineDefinition with the updated PipelineObjects, then ActivatePipeline to start the next run.
Pipeline Validation Before Promotion
Before promoting a Data Pipeline definition from staging to production, teams validate that all referenced fields, schedules, and resources are well-formed. ValidatePipelineDefinition returns errors and warnings without activating the pipeline, which prevents broken changes from running.
Call ValidatePipelineDefinition with the candidate PipelineObjects and return errors[] and warnings[] for review.
Custom Task Runner Coordination
Some teams run Data Pipeline activities on their own EC2 instances or on-premises hosts via the task runner protocol. PollForTask, ReportTaskProgress, and SetTaskStatus let custom task runners pull work, report heartbeat progress, and report success or failure.
Call PollForTask with a Worker Group, execute the returned activity, then call SetTaskStatus with FINISHED.
Agent-Assisted Migration Off Data Pipeline
An AI agent helping a team migrate off Data Pipeline can list every pipeline, fetch each definition for review, and deactivate ones that have been replaced by Step Functions or Glue Workflows. Through Jentic, the agent issues each operation as one structured call.
Call ListPipelines, then for each pipelineId call GetPipelineDefinition and DescribePipelines, and deactivate any pipeline tagged 'replaced=true'.
Key Endpoints
19 endpoints — jentic publishes the only available openapi specification for aws data pipeline, keeping it validated and agent-ready.
METHOD
PATH
DESCRIPTION
POST
/#X-Amz-Target=DataPipeline.CreatePipeline
Create a new pipeline
POST
/#X-Amz-Target=DataPipeline.PutPipelineDefinition
Set the pipeline definition
POST
/#X-Amz-Target=DataPipeline.ValidatePipelineDefinition
Validate a pipeline definition
POST
/#X-Amz-Target=DataPipeline.ActivatePipeline
Activate a pipeline
POST
/#X-Amz-Target=DataPipeline.DeactivatePipeline
Deactivate a pipeline
POST
/#X-Amz-Target=DataPipeline.DescribePipelines
Describe pipeline state
POST
/#X-Amz-Target=DataPipeline.PollForTask
Poll for the next task as a task runner
POST
/#X-Amz-Target=DataPipeline.CreatePipeline
Create a new pipeline
POST
/#X-Amz-Target=DataPipeline.PutPipelineDefinition
Set the pipeline definition
POST
/#X-Amz-Target=DataPipeline.ValidatePipelineDefinition
Validate a pipeline definition
POST
/#X-Amz-Target=DataPipeline.ActivatePipeline
Activate a pipeline
POST
/#X-Amz-Target=DataPipeline.DeactivatePipeline
Deactivate a pipeline
POST
/#X-Amz-Target=DataPipeline.DescribePipelines
Describe pipeline state
POST
/#X-Amz-Target=DataPipeline.PollForTask
Poll for the next task as a task runner
Jentic AI Readiness Score
This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.
A
AWS Data Pipeline
- AI-Aware (B-)61/100
99
Foundational Compliance63
Developer Experience & Jentic Compatibility38
AI-Readiness & Agent Experience94
Agent Usability50
Security100
AI DiscoverabilityPowered by JenticScoring Framework 1.0.0 | Scoring Engine 0.4.0
Show dimension breakdown
99
Foundational Compliance
Base layer of spec validity and structural soundness.
Grade: A+Signals: 4
97%
Lint Results
Aggregated quality score from linter diagnostics, weighted by severity.
100%
Resolution Completeness
Percentage of `$ref` references that resolve successfully.
100%
Specification Validity
Checks whether the API description parses successfully and conforms to its declared specification (e.g., OpenAPI).
100%
Structural Integrity
Structural correctness score based on schema issues using logarithmic dampening.
63
Developer Experience & Jentic Compatibility
Clarity, completeness, and ingestion readiness for developers and tooling.
Grade: B-Signals: 4
0%
Example Density
How richly the API is illustrated with examples.
100%
Example Validity
Percentage of examples that conform to their schemas.
50%
Response Coverage
Percentage of operations with complete response definitions (success, client error, server error).
100%
Tooling Readiness
Health of API ingestion, bundling, and resolution within Jentic pipelines.
38
AI-Readiness & Agent Experience
Semantic breadth, depth, and agent comprehension for AI systems.
Grade: FSignals: 4
52%
Description Coverage
Coverage of descriptions across API elements.
0%
Error Standardization
Coverage of RFC 9457 Problem Details for error responses.
100%
OperationId Quality
Coverage, uniqueness, and casing consistency of operationIds for AI inference.
0%
Summary Coverage
Coverage of summaries across operations/tags/info.
94
Agent Usability
Functional utility, complexity comfort, and AI orchestration readiness.
Grade: A+Signals: 1
94%
Complexity Comfort
Agent comfort level based on API operational and structural complexity.
50
Security
Trust, risk posture, and security compliance.
Grade: C-Signals: 1
50%
Authentication Strength
Average quality of security schemes based on authentication method strength (weakest link for OAuth2).
100
AI Discoverability
Findability, semantic richness, and reasoning readiness.
Grade: A+Signals: 1
100%
Descriptive Richness
Clarity and depth of descriptions across API elements.
Score it yourself
Every API in the directory is allowlisted, so you can re-score it with no key required.
npx @jentic/api-scorecard-cli score <openapi-url>Why Jentic?
What agents get from Jentic-routed access to this vendor.
Setup
Wiring AWS Data Pipeline by hand means implementing AWS Signature v4 request signing, resolving the regional datapipeline host, and setting the X-Amz-Target header to route each JSON action to the right operation. Through Jentic you install once, import AWS Data Pipeline from the API Directory, store the AWS credentials once, and your agent calls it.
Permission scoping
This API routes every call through a single endpoint with the action named in the request, covering operations like CreatePipeline, ValidatePipelineDefinition, and ActivatePipeline. Limit the agent to the operations it needs, such as DescribePipelines and ValidatePipelineDefinition, so DeactivatePipeline is not included unless you add it.
Credential isolation
Your AWS credentials are stored once, encrypted, by your own Jentic One instance and injected at execution time. They never enter the agent's prompt, logs, or context.
Intent-based discovery
Agents search Jentic by intent such as 'validate and activate a Data Pipeline' or 'describe pipeline state', and Jentic returns the matching AWS Data Pipeline operation with its input schema so the agent calls the right endpoint without browsing the AWS service reference.
Related APIs
Alternatives and complements available in the Jentic catalogue.
FAQs
Specific to using AWS Data Pipeline API through Jentic.
Why is there no official OpenAPI spec for AWS Data Pipeline?
AWS does not publish an OpenAPI specification. Jentic generates and maintains this spec so that AI agents and developers can call AWS Data Pipeline via structured tooling. It is validated against the live API and kept up to date. Get started with Jentic One, the self-hosted execution layer.
What authentication does the AWS Data Pipeline API use?
AWS Signature v4 (HMAC) with IAM permissions on datapipeline:* actions plus pass-through permissions for the AWS resources the pipeline uses (S3, EC2, EMR, RDS, Redshift). Jentic stores the AWS credentials in its vault and signs each request.
Can I activate and deactivate pipelines with the API?
Yes. ActivatePipeline starts the next scheduled execution and DeactivatePipeline halts further runs without deleting the pipeline. DescribePipelines reports current state.
What are the rate limits for the AWS Data Pipeline API?
Per-account, per-region TPS limits apply, and PollForTask is the rate-limiting operation for custom task runner fleets - back off when no task is returned. AWS treats Data Pipeline as legacy, so capacity may be lower than newer services.
How do I validate a pipeline definition through Jentic?
Search Jentic for 'validate a Data Pipeline definition before activating it', load the ValidatePipelineDefinition schema, and execute it with the pipelineId and PipelineObjects. Errors and warnings come back in structured arrays so you can fail closed before activation.
Should I use AWS Data Pipeline for new workloads?
AWS announced Data Pipeline as legacy in 2023 and recommends AWS Step Functions, AWS Glue Workflows, or Amazon MWAA (managed Airflow) for new work. The API remains supported for existing pipelines.
Can I limit what my agent is allowed to do with the AWS Data Pipeline API?
Yes. Because you run Jentic One yourself, your own rules decide which AWS Data Pipeline operations the agent may call, even though every request routes through a single endpoint with the action named inside it. You can allow read-and-check operations like DescribePipelines and ValidatePipelineDefinition while withholding state-changing ones such as ActivatePipeline, DeactivatePipeline, or CreatePipeline. The agent only ever reaches the operations you have explicitly granted, using the AWS credentials your instance holds.
GET STARTED