Hadoop Oozie学习笔记 Oozie不支持Sqoop问题解决

搭建起了Hadoop＋oozie+Sqoop,准备写个Oozie工作流,里面通过Sqoop从DB里导数据再写到HDFS.一切就绪,但却报错了.报错如下:

Error: E0701 : E0701: XML schema error, cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'sqoop'.

再去看日志,好家伙,错误一大把,但已经能否发现些什么.异常如下:

1. 2011-11-11 11:39:44,658  WARN V1JobsServlet:539 - USER[?] GROUP[users] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST http://localhost:11000/oozie/v1/jobs?action=start] error[E0701], E0701: XML schema error, cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'sqoop'.org.apache.oozie.servlet.XServletException: E0701: XML schema error, cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'sqoop'.   
2.     at org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:163)   
3.     at org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:74)   
4.     at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:92)   
5.     at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)   
6.     at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:281)   
7.     at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)   
8.     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)   
9.     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)   
10.     at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)   
11.     at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)   
12.     at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)   
13.     at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)   
14.     at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)   
15.     at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)   
16.     at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)   
17.     at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)   
18.     at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)   
19.     at java.lang.Thread.run(Thread.java:662)   
20. Caused by: org.apache.oozie.DagEngineException: E0701: XML schema error, cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'sqoop'.   
21.     at org.apache.oozie.DagEngine.submitJob(DagEngine.java:137)   
22.     at org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:159)   
23.     ... 17 more   
24. Caused by: org.apache.oozie.command.CommandException: E0701: XML schema error, cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'sqoop'.   
25.     at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:185)   
26.     at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:61)   
27.     at org.apache.oozie.command.XCommand.call(XCommand.java:257)   
28.     at org.apache.oozie.DagEngine.submitJob(DagEngine.java:125)   
29.     ... 18 more   
30. Caused by: org.apache.oozie.workflow.WorkflowException: E0701: XML schema error, cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'sqoop'.   
31.     at org.apache.oozie.workflow.lite.LiteWorkflowAppParser.validateAndParse(LiteWorkflowAppParser.java:120)   
32.     at org.apache.oozie.workflow.lite.LiteWorkflowLib.parseDef(LiteWorkflowLib.java:47)   
33.     at org.apache.oozie.service.LiteWorkflowAppService.parseDef(LiteWorkflowAppService.java:46)   
34.     at org.apache.oozie.service.LiteWorkflowAppService.parseDef(LiteWorkflowAppService.java:41)   
35.     at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:95)   
36.     ... 21 more   
37. Caused by: org.xml.sax.SAXParseException: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'sqoop'.   
38.     at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)   
39.     at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:131)   
40.     at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:384)   
41.     at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:318)   
42.     at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator$XSIErrorReporter.reportError(XMLSchemaValidator.java:417)   
43.     at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.reportSchemaError(XMLSchemaValidator.java:3182)   
44.     at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1927)   
45.     at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:705)   
46.     at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:400)   
47.     at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2755)   
48.     at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)   
49.     at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)   
50.     at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)   
51.     at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)   
52.     at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)   
53.     at com.sun.org.apache.xerces.internal.jaxp.validation.StreamValidatorHelper.validate(StreamValidatorHelper.java:144)   
54.     at com.sun.org.apache.xerces.internal.jaxp.validation.ValidatorImpl.validate(ValidatorImpl.java:111)   
55.     at javax.xml.validation.Validator.validate(Validator.java:127)   
56.     at org.apache.oozie.workflow.lite.LiteWorkflowAppParser.validateAndParse(LiteWorkflowAppParser.java:106)   
57.     ... 25 more

错误大概就是xml不支持自定义标签sqoop.这里我在Oozie中需要使用Sqoop从DB中导入数据,所以在workflow.xml中进行了如下配置:

1.       <action name="db_export">
2.       <sqoop xmlns="uri:oozie:sqoop-action:0.2">  
3.       <job-tracker>${job_tracker}</job-tracker>  
4.       <name-node>${name_node}</name-node>  
5.       <prepare>  
6.         <delete path="${wf_job_base_path}/${wf:id()}/db_export"/>  
7.       </prepare>  
8.       <configuration>  
9.         <property>  
10.           <name>mapred.job.queue.name</name>  
11.           <value>${queue_name}</value>  
12.         </property>  
13.       </configuration>  
14.       <arg>import</arg>  
15.       <arg>-D</arg>  
16.       <arg>mapred.output.compress=false</arg>  
17.       <arg>--connect</arg>  
18.       <arg>jdbc:mysql://${db_hostname}/${db_name}</arg>  
19.       <arg>--query</arg>  
20.       <arg>${db_banner_query}</arg>  
21.       <arg>--target-dir</arg>  
22.       <arg>${wf_job_base_path}/${wf:id()}/db_export</arg>  
23.       <arg>--num-mappers</arg>  
24.       <arg>${sqoop_export_mappers_num}</arg>  
25.       <arg>--username</arg>  
26.       <arg>${db_user}</arg>  
27.       <arg>--password</arg>  
28.       <arg>${db_password}</arg>  
29.       <arg>--as-sequencefile</arg>  
30.       <arg>--class-name</arg>  
31.       <arg>${db_class_name}</arg>  
32.     </sqoop>  
33.     <ok to="sqoop_export_done"/>  
34.     <error to="fail"/>  
35.   </action>

常识告诉我,自定义标签需要进行相关命名.看这边sqoop引入的是uri:oozie:sqoop-action:0.2,所以应该要有oozie-sqoop-action-0.2.xsd文件.于是去$OOZIE_HOME/lib/oozie-clinet-x.x.jar中

并未看到该文件.于是按照http://archive.cloudera.com/cdh/3/oozie/DG_SqoopActionExtension.html的配置新建该文件并放入该jar中.继续运行,但还是报错.同时随着对自定义标签的了解以及看了这篇文章后http://www.infoq.com/cn/articles/ExtendingOozie,我发现需要相关的ActionSuppoert支持,而在oozie-core中并未找到org.apache.oozie.action.hadoop.SqoopActionExecutor这个类.然后通过搜索才发现,之前是下载yahoo的oozie,而只有

cloudera才对oozie支持使用sqoop.于是去cloudera下载了oozie-2.3-cdhu30.tar.gz,并从中找到相关的xsd和ActionSupport.再直接运行,OK,问题解决!看来用上了cloudera,以后啥都要先围着它去转了.下载地址:http://archive.cloudera.com/cdh/3/oozie-2.3.2-cdh3u2.tar.gz