kafka

I have noticed when ingesting backlog(older timestamped data) that the "Messages per minute" line graph and "sources" data do not line up.

The Messages per minute appear to be correct for the ingest rate, but the sources breakdown below it only show messages for each type from within the time window via timestamp. This means in the last hour if you've ingested logs from 2 days ago, the data is

https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/servicebus/Azure.Messaging.ServiceBus/MigrationGuide.md

Is your feature request related to a problem? Please describe.

A user in the community Slack wanted to filter out/select items from a JSON array of objects. Given records in the following format:

'[{"type": "AAA", "timestamp": "2021-09-27"}, {"type": "BBB", "timestamp": "2021-09-27"}, {"type": "AAA", "t

Under the hood, Benthos csv input uses the standard encoding/csv packages's csv.Reader struct.

The current implementation of csv input doesn't allow setting the LazyQuotes field.

We have a use case where we need to set the LazyQuotes field in order to make things work correctly.

This comment says that the message ID is optional,
but for SQL transport it is a mandatory attribute,
in turn it causes misunderstanding?

Is it possible to fix it or did I get something wrong?

https://github.com/ThreeDotsLabs/watermill/blob/b9928e750ba673cf93d442db88efc04706f67388/message/message.go#L20
https://github.com/ThreeDotsLabs/watermill/blob/b9928e750ba673cf93d442db88efc04706f6

Version & Environment

Redpanda version: (use rpk version): v21.11.10
Kafka client: franz-go v1.4.0