current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
Join the Stack Overflow Community
Stack Overflow is a community of 6.3 million programmers, just like you, helping each other.
Join them; it only takes a minute:
Sign up
up vote 0 down vote favorite
1

I am new to apache kafka and apache spark. I want to integrate the kafka with my angularjs code. Basically I want to make sure that, when a user click on any link or searches anything on my website, then the those searches and clicks should be triggered as an event and send it to the kafka data pipe for the use of analytics. My question is how can I integrate frontend code which is in angular.js to apache kafka? Can I send the searches and click stream data directly to apache spark using kafka pipeline or do I need to send those data to kafka and apache spark will do polling to kafka server and receive the data in batches?

share|improve this question
up vote 0 down vote

I don't think (just cannot find at glance) there is Kafka client for front-end JavaScript. I cannot actually imagine stable setup when millions of producers (each client's browser) writing to the same Kafka topic. What you need to do in Angular, is to call your server side function to log your events in Kafka. Server side code may be written in a bunch of languages, including JavaScript for node.js. Please take a look for available clients at Kafka Documentation

share|improve this answer
    
I do see, Kafka-node for node.js, however I was looking for client plugin for angular.js. Since kafka is scalable, so I wanted to make sure that, each client (i.e. producer) connect to the same topic. – Vikas Pandey 23 hours ago
    
Kafka built for sustainable stream of data entering topic and not for millions of opened/closed connections, what would be in client side acquisition scenario. You can prepare pretty simple feasibility test of this. Results will be interesting though. – Robert Navado 23 hours ago
    
So, basically there is no way, we can implement kafka client using angular.js . We need to send the REST calls from angular to the route handlers in node.js, which in turn will act as the kafka producer. Will it send the data using the kafka data pipeline to apache spark which will be consumer or will it send the data using kafka data pipeline to apache kafka server? – Vikas Pandey 21 hours ago
    
Actually there are some websocket implementations that you could invoke from your controllers in Angular, that way you have full-duplex communication channels over a single TCP connection. But you still need a back-end to set up the websocket and delegate to Kafka. – Nick Vanderhoven 8 hours ago

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.