Skip to content
#

spark-streaming

Here are 596 public repositories matching this topic...

fennuzhichui
fennuzhichui commented Jan 2, 2020

how to define java8 when submit application use spark-submit


name: Bug report/Feature request/Question
about: Create a report to help us improve
title: ''
label: bug/enhancement/question
assignees: ''

Environment:

  • Java version:
  • Scala version:
  • Spark version:
  • PyTorch and Python version:
  • OS and version:

Checklist:

  • Did you check if your bug/feature/
imback82
imback82 commented Mar 19, 2019

We create multiple jars during our builds to accommodate multiple versions of Apache Spark. In the current approach, the implementation is copied from one version to another and then necessary changes are made.

An ideal approach could create a common directory and extract common classes from duplicate code. Note that even if class/code is exactly the same, you cannot pull out to a common clas

garyelephant
garyelephant commented Mar 18, 2019

FAQ 1. Waterdrop开发者自己开发插件时,是否需要了解Waterdrop代码,是否需要把代码写到Waterdrop项目里?

开发者开发的插件,与waterdrop项目可以完全没有关系,不需要把你的插件代码写到waterdrop项目里面。
插件可以是一个完全独立的项目,在里面你用java,scala,maven,sbt,gradle,都随你。
这也是我们建议开发者开发插件的方式。


FAQ 2. 以集群模式(cluster)运行waterdrop,提示找不到:plugins.tar.gz

使用cluster模式提交前,需要您先执行如下命令:

# 备注: 预计下一个版本发布时,v1.2.3 我们会支持插件目录自动打包,无需再执行此命令。
tar zcvf plugins.tar.gz plugins

将插件目

andrewjw1995
andrewjw1995 commented May 16, 2018

The documentation file appears to have been generated with no space between the hashes and the header text. This is causing the headers to not display correctly, and is difficult to read. See below for an example of with and without the space:

##

Mobius API Documentation


###Microsoft.Spark.CSharp.Core.Accumulator</

letlife
letlife commented Aug 1, 2019

com.alibaba.fastjson.JSONException: syntax error, error
at com.alibaba.fastjson.parser.DefaultJSONParser.parseObject(DefaultJSONParser.java:600)
at com.alibaba.fastjson.parser.DefaultJSONParser.parse(DefaultJSONParser.java:1394)
at com.alibaba.fastjson.parser.DefaultJSONParser.parse(DefaultJSONParser.java:1360)
at com.alibaba.fastjson.JSON.parse(JSON.java:165)

carlbrochu
carlbrochu commented Apr 18, 2019

Describe the bug
Some utilities project are duplicated between singular (utility) and plural versions (utilities). Let's align to plural versions

To Reproduce
Steps to reproduce the behavior:

  1. Go to Services\DataX.Utilities folder
  2. Notice duplicate folders like 2 cosmosdb utils

Expected behavior
One dll per area

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

  • Updated Feb 1, 2019
  • TypeScript
HeartSaVioR
HeartSaVioR commented Sep 6, 2017

We have a [document|https://github.com/hortonworks/streamline/blob/master/docs/REST-API.md] for REST API but this looks like out of sync and also easy to be out of sync.

It would be better if we could document API to the source code of each endpoint, and apply swagger or so to make it some pages.
(Registry project uses swagger.)

Improve this page

Add a description, image, and links to the spark-streaming topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the spark-streaming topic, visit your repo's landing page and select "manage topics."

Learn more

You can’t perform that action at this time.