Oozie工作流框架使用指南PPT課件

工作流框架使用指南 2013年10月 1 SystemInstallationJobDesignOozieOverviewWorkFlowDesignCoordinatorDesignReference Contents 2 RequirementsClouderaManagerCDHOozieInstallConfigHueInstallConfig 2items BasicOperation SystemInstallation 3 ActionOverviewAnexecution computationtask Map Reducejob Pigjob ashellcommand Itcanalsobereferredastaskor actionnode JobDesignerJobDesignExampleFsActionMapReduceAction JobDesign 4 OozieAworkflowschedulersystemtomanageApacheHadoopjobsWorkFlowWorkflowjobsareDirectedAcyclicalGraphs DAGs ofactionsCoordinatorCoordinatorjobsarerecurrentworkflowjobstriggeredbytime frequency anddataavailabilty IntegratedMR FS Email Shell Ssh Hive Pig Sqoop Distcp JavaScalableReliableExtensible OozieOverview 5 DoesnotsupportcirclesWorkflowNodesControlFlowNodesWorkflowActionNodesWorkFlowRecoveryoozie wf rerun failnodesoozie wf rerun skip nodes WorkFlowDesign 6 WorkflowdefinitionscanbeparameterizedWhenworkflownodeisexecutedbyOoziealltheELsareresolvedintoconcretevaluesELexpressionscanbeusedintheconfigurationvaluesofactionanddecisionnodesWorkflowJobProperties orParameters ExpressionLanguageFunctions ParameterizationofWorkflows 7 Example Start MapReduce1 輸出 500行 MapReduce2 End 是 8 Coordinatorexecuteworkflowjobs RecurrentInterdependentCoordinatorBased TimeintervalsDataavailabilityTimeintervalsand ordataavailability CoordinatorOverview 9 CoordinatorArchtechture 10 Datetime FrequencyandTime PeriodCoordinatorActionParameterizationofCoordinatorCoordinatorDesign CoordinatorDesign 11 定期創(chuàng)建文件夾開始時間 結(jié)束時間 頻率 每3分鐘一次文件夾名字為標定時間格式化yyyy MM ddTHH mm Example 12 O