recipe-patterns

name recipe-patterns description Use when creating, configuring, or running any Dataiku recipe (prepare, join, group, sync, python) including data cleaning, formulas, and GREL Dataiku Recipe Patterns Reference patterns for creating different recipe types via the Python API. Before Writing Code MANDATORY : Read the relevant reference file before writing any recipe code. GREL formulas → read references/grel-functions.md first Prepare steps → read references/processors.md first Joins → read references/join-recipe.md first Grouping → read references/group-recipe.md first Python recipes → read references/python-recipe.md first Sync recipes → read references/sync-recipe.md first Date handling → read references/date-operations.md first Pitfalls index → references/pitfalls.md (recipe-type reference files also have a Pitfalls section at the top) Do NOT rely on general knowledge for GREL functions or API methods. Dataiku GREL differs from OpenRefine GREL and other variants. Always verify function names against the reference. Recipe Type Decision Table Recipe Type Use When Key Method Prepare Column transforms, filtering, formula columns, renaming, data cleaning project.new_recipe("prepare", ...) Join Combining datasets on key columns (LEFT, INNER, RIGHT, OUTER) project.new_recipe("join", ...) Group Aggregations: sum, count, avg, min, max, stddev, etc. project.new_recipe("grouping", ...) Sync Copying data between connections (e.g., to a data warehouse) project.new_recipe("sync", ...) Python Custom transformations not possible with visual recipes project.new_recipe("python", ...) Universal Builder Pattern Every recipe follows the same create-configure-run lifecycle: # 1. Create via builder builder = project.new_recipe( "<type>" , "<recipe_name>" ) builder.with_input( "<input_dataset>" ) builder.with_new_output( "<output_dataset>" , "<connection>" ) # creates output dataset recipe = builder.create() # 2. Configure settings settings = recipe.get_settings() # ... recipe-specific configuration ... settings.save() # 3. Apply schema updates schema_updates = recipe.compute_schema_updates() if schema_updates.any_action_required(): schema_updates.apply() # 4. Run and check job = recipe.run(no_fail= True ) state = job.get_status()[ "baseStatus" ][ "state" ] # "DONE" or "FAILED" After Running Any Recipe Always sample the output and verify the result before reporting success. Silent data issues (wrong values, all nulls, unexpected types) are common. from helpers.export import sample rows = sample(client, "PROJECT_KEY" , "output_dataset" , 5 ) for r in rows: print (r) Always Remember Call settings.save() after configuration changes Call compute_schema_updates().apply() for visual recipes Call recipe.run(no_fail=True) to execute (already waits for completion) Check job.get_status()["baseStatus"]["state"] for "DONE" or "FAILED" Sample and verify the output data before reporting success Tested Patterns Copy-paste patterns that have been validated against a live Dataiku instance: patterns/bin-numeric-column.py — Bin a string numeric column into ranges patterns/calculated-columns.py — Common GREL formula patterns patterns/filter-and-clean.py — Data cleaning pipeline Detailed References Recipe types: references/prepare-recipe.md — Prepare recipe builder, add_processor_step() API references/join-recipe.md — Join configuration, multi-table joins, column selection references/group-recipe.md — Aggregation flags, output naming, type compatibility references/sync-recipe.md — Sync recipe pattern references/python-recipe.md — Python recipe with set_code Data preparation: references/processors.md — All processor types with parameters and complete example references/grel-functions.md — Full GREL function table and formula syntax references/date-operations.md — DateParser, DateFormatter, datePart examples Troubleshooting: references/pitfalls.md — Index of all pitfalls (details are inline in each reference file)