current community

your communities

more stack exchange communities

Stack Exchange
tour help
careers 2.0
Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 3 down vote favorite

While running s3distcp from S3 to HDFS:

sudo -u hdfs hadoop jar /usr/lib/hadoop/lib/s3distcp.jar --src s3n://workAAAA-KKKK-logs/production-logs/Log-XXXX-click/Log-XXXXX-click-2013-03-27_06-21-19_i-7XXb2x39_00037.gz --dest hdfs:///test/

I get the following exception.

Is there something wrong with my path syntax (s3n:// ; hdfs:///) ? Has anyone encountered this issue before?

Thank you

13/04/04 12:10:52 INFO s3distcp.S3DistCp: Using output path 'hdfs:/tmp/96a8e57b-4c68-406c-b4ca-bf212de12d93/output'

13/04/04 12:10:53 INFO s3distcp.FileInfoListing: Opening new file: hdfs:/tmp/96a8e57b-4c68-406c-b4ca-bf212de12d93/files/1 Exception in thread "main" java.lang.IllegalArgumentException: Can not create a Path from an empty string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:91) at org.apache.hadoop.fs.Path.(Path.java:99) at org.apache.hadoop.fs.Path.(Path.java:58) at com.amazon.external.elasticmapreduce.s3distcp.FileInfoListing.getOutputFilePath(FileInfoListing.java:155) at com.amazon.external.elasticmapreduce.s3distcp.FileInfoListing.add(FileInfoListing.java:111) at com.amazon.external.elasticmapreduce.s3distcp.FileInfoListing.add(FileInfoListing.java:78) at com.amazon.external.elasticmapreduce.s3distcp.S3DistCp.createInputFileListS3(S3DistCp.java:122) at com.amazon.external.elasticmapreduce.s3distcp.S3DistCp.createInputFileList(S3DistCp.java:60) at com.amazon.external.elasticmapreduce.s3distcp.S3DistCp.run(S3DistCp.java:529) at com.amazon.external.elasticmapreduce.s3distcp.S3DistCp.run(S3DistCp.java:216) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at com.amazon.external.elasticmapreduce.s3distcp.Main.main(Main.java:12) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

share|improve this question
2  
This might not be the issue here but the source path should be a directory path and not a file path. –  Amar Apr 4 '13 at 12:31
1  
Are you sure hdfs:/// shouldn't be hdfs://? –  Quetzalcoatl Apr 4 '13 at 12:54
    
@Amar You are completely right, now it works! –  bocse Apr 4 '13 at 13:06
    
@Quetzalcoatl hdfs:/// is correct (three slashes). –  bocse Apr 4 '13 at 13:07
    
@bocse glad it helped. –  Amar Apr 4 '13 at 19:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.