hadoop
Here are 2,111 public repositories matching this topic...
Can documentation be added on how to unit test workflows using the luigi mock module ?
Couldn't find any examples in the lugi/examples folder either
We have a value_at_quantile function for QDigest types. Occasionally, it's useful to also retrieve a quantile given a value (say, for X, which percentile in the QDigest does it fall under).
The signature would look like the following:
quantile_at_value(qdigest(T), DOUBLE) -> T
Where T is one of DOUBLE, REAL or BIGINT.
-
Updated
Jun 4, 2020 - Jupyter Notebook
Page
https://docs.alluxio.io/os/user/stable/en/deploy/Running-Alluxio-On-Docker.html
Summary
Two environment variables are not documented, and have negative impact when running Alluxio purely as a local disk cache via docker containers.
- ALLUXIO_WORKER_TIEREDSTORE_LEVEL0_DIRS_PATH
- ALLUXIO_RAM_FOLDER
If neither of these environment variables is set, and the docker container i
Spark 2.3 officially support run on kubernetes. While our guide of "Run on Kubernetes" is still based on a special version of Spark 2.2, which is out of date. We need to:
- update that document to Spark 2.3
- release the corresponding docker images.
-
Updated
Jun 3, 2020 - Java
Validation should be added to directed fields in schemas. This will be done as part of work for version 2 as adding in validation would cause breaking changes.
-
Updated
Apr 1, 2019 - Java
Along with the tests:
/ibis/ibis/expr/api.py:318: DeprecationWarning: ibis.nanosecond is deprecated, use ibis.interval(nanoseconds=n) instead
warnings.warn(msg.format(name), DeprecationWarning)
/ibis/ibis/expr/api.py:318: DeprecationWarning: ibis.microsecond is deprecated, use ibis.interval(microseconds=n) instead
warnings.warn(msg.format(name), DeprecationWarning)
/ibis/i
CHAR is mapped to Kudu string
https://github.com/prestosql/presto/blob/ec549ac8a1192b18c3667203974f453d5fe5a9fa/presto-kudu/src/main/java/io/prestosql/plugin/kudu/TypeHelper.java#L88-L89
but char(3) value ab⎵ is not persisted as such, it's stored as ab instead.
Waterdrop FAQ/常见问题
FAQ 1. Waterdrop开发者自己开发插件时,是否需要了解Waterdrop代码,是否需要把代码写到Waterdrop项目里?
开发者开发的插件,与waterdrop项目可以完全没有关系,不需要把你的插件代码写到waterdrop项目里面。
插件可以是一个完全独立的项目,在里面你用java,scala,maven,sbt,gradle,都随你。
这也是我们建议开发者开发插件的方式。
FAQ 2. 以集群模式(cluster)运行waterdrop,提示找不到:plugins.tar.gz
使用cluster模式提交前,需要您先执行如下命令:
# 备注: 预计下一个版本发布时,v1.2.3 我们会支持插件目录自动打包,无需再执行此命令。
tar zcvf plugins.tar.gz plugins
将插件目
-
Updated
Jun 3, 2020 - Python
-
Updated
May 12, 2020 - Java
Not sure if it's worth adding to the documentation but I found I had to add the following to the end of the fstab command otherwise the system would not boot cleanly.
,x-systemd.mount-timeout=30,_netdev
The line in fstab now looks like
mfsmount /mnt/mymnt fuse mfssubfolder=mydir,allow_other,x-systemd.mount-timeout=30,_netdev
I think it was trying to mount mooseFS before it had fini
-
Updated
May 15, 2020 - Java
The Hadoop official document recommends to set dfs.encrypt.data.transfer.cipher.suites to AES.
"AES offers the greatest cryptographic strength and the best performance."
Setting it to AES/CTR/NoPadding activates AES encryption. By default, this is unspecified, so AES is not used.
I wonder if
-
Updated
Jun 3, 2020 - Shell
- 安装linkis jobtypes
按照官方安装文档进行自动化安装,执行sh install.sh最后一步报错:{"error":"Missing required parameter 'execid'."}。并没有看到文档中所说的“如果安装成功最后会打印:{"status":"success"}”,但是能在azkaban的/plugins/jobtypes目录下看到已经安装好的linkis任务插件。通过排查在安装脚本最后一步会去调用"curl http://azkaban_ip:executor_port/executor?action=reloadJobTypePlugins"进行插件的刷新。重启azkaban executor日志中看到已经加载了插件的信息 `INFO [JobTypeManager][Azkaban] Loaded jobtype linkis
-
Updated
Jul 25, 2018 - Python
Useful for source of truth tony version for an application.
Improve this page
Add a description, image, and links to the hadoop topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the hadoop topic, visit your repo's landing page and select "manage topics."


Alexnet implementation in tensorflow has incomplete architecture where 2 convolution neural layers are missing. This issue is in reference to the python notebook mentioned below.
https://github.com/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/notebooks/3_neural_networks/alexnet.ipynb