data-engineering
Here are 1,116 public repositories matching this topic...
-
Updated
Feb 3, 2022
-
Updated
Jan 2, 2022
-
Updated
Jan 25, 2022
Current behavior
You get an error if you try to upload the same file name
azure.core.exceptions.ResourceExistsError: The specified blob already exists.
RequestId:5bef0cf1-b01e-002e-6
Proposed behavior
The task should take in an overwrite argument and pass it to [this line](https://github.com/PrefectHQ/prefect/blob/6cd24b023411980842fa77e6c0ca2ced47eeb83e/src/prefect/
Describe the bug
data docs columns shrink to 1 character width with long query
To Reproduce
Steps to reproduce the behavior:
- make a batch from a long query string
- run validation
- render result to data docs
- See screenshot
<img width="1525" alt="Data_documentation_compiled_by_Great_Expectations" src="https://user-images.githubusercontent.com/928247/103230647-30eca500-4
New Source: Mambu
Tell us about the new connector you’d like to have
- Which source and which destination?
Source: mambu (https://api.mambu.com/)
Destination: S3, RDS, Redshift - Do you need a specific version of the underlying data source e.g: you specifically need support for an older version of the API or DB?
Want to support for since 1.0.
Describe the context around this new connector
- Why do yo
Under the hood, Benthos csv input uses the standard encoding/csv packages's csv.Reader struct.
The current implementation of csv input doesn't allow setting the LazyQuotes field.
We have a use case where we need to set the LazyQuotes field in order to make things work correctly.
The current DynamoDB implementation does sequential gets (https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/online_stores/dynamodb.py#L163)
Possible Solution
A better approach is to do some multi-get operation or at least run these queries in parallel and collect the results.
-
Updated
Feb 7, 2022 - Python
In the documentation Hooks page under the Action file section, the hook.type description should be updated using the current available hook types and a closing parenthesis.
-
Updated
Feb 2, 2022
When we show data for a metric, we currently don't include the current day's worth of data. For users just getting set up, they may only have events from today, and want to test out if the query is working, and by excluding events from 'today', they can't see results.
TODO:
- In
packages/back-end/src/services/experiments.tson line329, instead of using the current date as the value
-
Updated
Aug 2, 2021 - JavaScript
link to our youtube channel (add link at the top)
https://docs.ploomber.io/en/latest/videos.html
more videos available in our youtube channel
also, if there's a way to add a link that subscribes people, that's grat
-
Updated
Feb 7, 2022 - Jupyter Notebook
-
Updated
Dec 31, 2021
-
Updated
Feb 1, 2022 - Jupyter Notebook
-
Updated
Mar 9, 2020 - Python
if they are not class methods then the method would be invoked for every test and a session would be created for each of those tests.
`class PySparkTest(unittest.TestCase):
@classmethod
def suppress_py4j_logging(cls):
logger = logging.getLogger('py4j')
logger.setLevel(logging.WARN)
@classmethod
def create_testing_pyspark_session(cls):
return Sp
-
Updated
Feb 3, 2022
Background
This thread is borne out of the discussion from #968 , in an effort to make documentation more beginner-friendly & more understandable.
One of the subtasks mentioned in that thread was to go through the function docstrings and include a minimal working example to each of the public functions in pyjanitor.
Criteria reiterated here for the benefit of discussion:
It sh
Is your feature request related to a problem? Please describe.
If there are issues during ElasticSearch calls, we currently do not see any logs.
Describe the solution you'd like
Let's add logging in the backoff https://github.com/open-metadata/OpenMetadata/blob/main/catalog-rest-service/src/main/java/org/openmetadata/catalog/events/AbstractEventPublisher.java#L56 for any exceptions th
-
Updated
Feb 1, 2022 - Python
-
Updated
Jun 2, 2021
-
Updated
Mar 5, 2020 - Python
-
Updated
Feb 2, 2022
-
Updated
Feb 7, 2022 - Python
-
Updated
Nov 6, 2021 - Ruby
Improve this page
Add a description, image, and links to the data-engineering topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the data-engineering topic, visit your repo's landing page and select "manage topics."
The Mixed Time-Series chart type allows for configuring the title of the primary and the secondary y-axis.
However, while only the title of the primary axis is shown next to the axis, the title of the secondary one is placed at the upper end of the axis where it gets hidden by bar values and zoom controls.
How to reproduce the bug