current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
Code Review Stack Exchange is a question and answer site for peer programmer code reviews. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top
up vote -4 down vote favorite

I am a beginner in spark Scala . Given rdd1 and rdd2 

val rdd1 = sc.parallelize(Seq(
 |                ("key1", 1),
 |                ("key2", 6),
 |                ("key3", 5)))

val rdd2 = sc.parallelize(Seq(
 |                ("key1", 1),
 |                ("key2", 6),
                  ("key4", 7)))

I need to identify which value was deleted, which value was inserted and which value was modified I need a to add a column indicating this statue. I am expecting a result of the form

(key1,1,5,"modif")
(key2,6,6, " ")
(key3,5, null, "del")
(Key4,null,"insert")

here is my code

val grouped = rdd1.cogroup(rdd2)

def f(x: (String, (Iterable[Int], Iterable[Int]))) = {
 if(x._2._1 == null && x._2._2  != null)  x+ "insert"
 if(x._2._1 != null && x._2._1== null)  x+ "suppression"
 else x + "modif"
}

But It seems not getting inside the two first "if conditions", the code returns a new column containing "modif" at each component.

Thank you for your precious help!!!

share|improve this question

closed as off-topic by Pimgd, Jamal♦ Mar 15 at 10:16

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions containing broken code or asking for advice about code not yet written are off-topic, as the code is not ready for review. After the question has been edited to contain working code, we will consider reopening it." – Pimgd, Jamal
If this question can be reworded to fit the rules in the help center, please edit the question.
    
Hi there, welcome to Code Review! Here we review working, existing code to make it better. Right now you just have data that you want to transform from one shape to another - basically, you don't have any code written yet! So we can't review your code and this question will get closed. Once you do have working code, we'll be happy to review it. – Pimgd Mar 15 at 9:26
    
Thank you for your answer I just added my code – user100289 Mar 15 at 10:49
    
Welcome to Code Review! I'm afraid this question does not match what this site is about. Code Review is about improving existing, working code. Code Review is not the site to ask for help in fixing or changing what your code does. Once the code does what you want, we would love to help you do the same thing in a cleaner way! Please see our help center for more information. – Mast Mar 15 at 14:04