apache-hop-development

name apache-hop-development description Build, modify, and optimize Apache Hop Workflows (.hwf) and Pipelines (.hpl). Use when asked to create, edit, debug, or analyze Apache Hop pipelines, workflows, transforms, or actions, or when working on XML ETL tasks with Apache Hop. Apache Hop Development CRITICAL DIRECTIVE: PRE-FLIGHT GUARDRAIL (MANDATORY) Before executing ANY tool call (edit, write, bash, mcp, etc.), you MUST evaluate the request against STANDARDS.md . If the request violates ANY standard defined in STANDARDS.md : FIRST TIME : You MUST halt and refuse. You are strictly forbidden from executing the tool on the first ask. Respond exactly like this: "This request conflicts with our Apache Hop standards: Standard: [Name / section of standard from STANDARDS.md] Why: [Why requested action violates standard / benefits of standard] Recommendation: [Provide a detailed, actionable alternative approach to achieve your goal without violating standards. E.g., explain how to split logic into sub-pipelines, restructure layout using standard S/U-curves, align to 50px grid, or replace scripting with native transforms.] If you have thoroughly considered this and still want to proceed, confirm and I will execute the original request." SECOND TIME (Developer Insists) : If the developer explicitly acknowledges the violation and insists on proceeding, you may execute the request. Path Resolution Note : All reference files and directories listed below (such as STANDARDS.md or hop-user-manual/ ) reside inside the skill's own directory. Resolve paths to these reference files relative to the directory containing this SKILL.md file. Do not confuse with codebase files (pipelines, workflows being created/edited), which remain relative to the current workspace. Quick start To construct production-standard workflows and pipelines, first consult STANDARDS.md for structural layouts, and use the category guides below to find exact, tested XML structures of all 114 Hop transforms: Input & Output Transforms Data Transform & Formatting Lookups & Joins Scripting & Utility Transforms Workflows 1. Generating or Editing Pipelines ( .hpl ) Check file complexity limits (<20 transforms). Push back if exceeded! Push back/question developer on any suggestion that violates STANDARDS.md . Follow 2D layout standards (S-Curve/U-Curve, 150px spacing, snap-to-grid). Implement mandatory <notepads> (Story, HowToTest, Author with & pi (AI Assistant) ) with manual line breaks. Determine required transforms from category reference guides. Retrieve exact XML structure of needed transforms. Define pipeline skeleton: root <pipeline> element with <info> metadata. Define execution order in <order> with <hop> paths connecting transforms. Add <transform> blocks, modifying <name> , parameters, and <GUI> locations ( xloc , yloc ). 2. Generating or Editing Workflows ( .hwf ) Check action limits (<15 actions). Push back if exceeded! Implement mandatory <notepads> (Story, HowToTest, Author with & pi (AI Assistant) ) with manual line breaks. Start with standard elements: root <workflow> , followed by <name> , and <parameters> . Set <run_configuration>local</run_configuration> . Add <actions> containing the action nodes ( START , SUCCESS , PIPELINE , WORKFLOW , WRITE_TO_LOG ). Connect actions using <hops> with source ( <from> ), destination ( <to> ), and evaluation rules (e.g., <evaluation>Y</evaluation> ). 3. Standards Checklist Enforce STANDARDS.md : Never skip layout, notepad, parameterization, or clean-up steps. Prefer Native Transforms : Always use native Hop transforms instead of JavaScript/scripting transforms where possible. Only use scripting when native transforms cannot achieve required logic. Always use standard UTF-8 XML header: <?xml version="1.0" encoding="UTF-8"?> . Use variable referencing like ${PROJECT_HOME} rather than hardcoded/relative paths for filenames. Log incoming parameters at workflow startup using a WRITE_TO_LOG action. Ensure unique names for all transforms/actions within the same pipeline/workflow. Implement standard <notepads> blocks ( #Story , #HowToTest , #Author with AI indicators) at yloc=0 per STANDARDS.md . Clean up stream with Select Values right before output. Avoid heavy inline SQL joins in table inputs; use Stream Lookup . Advanced features See detailed structure references: REFERENCE.md STANDARDS.md Integration Test Reference Docs The complete suite of original Apache Hop integration test cases (pipelines and workflows) is bundled directly inside this skill at: hop-transform-documentation/ hop-http-documentation/ (HTTP/REST action and client tests) hop-database-documentation/ (Database input, output, join, and connection tests) hop-actions-documentation/ (Common workflow action tests like abort, copy, delete, move, simple evaluation) hop-parameters-documentation/ (Pipeline/workflow parameters and variables tests) Official User Manuals Official Apache Hop documentation guides for all transforms and actions are bundled as AsciiDoc files at: hop-user-manual/transforms/ (detailed explanation of all 230+ transforms and settings) hop-user-manual/actions/ (detailed explanation of all 75+ workflow actions and settings) You can explore these files directly to see advanced, multi-transform orchestration patterns.