Join the Stack Overflow Community
Stack Overflow is a community of 6.4 million programmers, just like you, helping each other.
Join them; it only takes a minute:
Sign up

I have a SaveQueue() task which I use for saving images to the database:

async Task SaveQueue()
{
    for (;;)
    {
        try
        {
            await Task.Delay(7500);
            dbUpload.Visibility = Visibility.Visible;
            dbUploadProgress.Value = 0;
            var q = new ObservableCollection<Picture>(manager.Queue.ToList().AsEnumerable());
            manager.Queue.Clear();
            var imgcount = q.Count();
            using (var pc = new PicsContext())
            {
                for (int i = 0; i < imgcount; i++)
                {
                    var tags = q[i].GetTags();
                    try
                    {
                        pc.Sites.Attach(q[i].Site); // I'm actually not sure how to use this. 
                    }
                    catch { }
                    var curUrl = q[i].Url;
                    if (pc.Pictures.FirstOrDefault(o => o.Url == curUrl) != null)
                    {
                        await Task.Delay(1);
                        dbUploadProgress.Value = (i + 1) / (double)imgcount;
                        dbUploadText.Text = "Skipping images...";
                        continue;
                    }
                    foreach (var t in tags)
                    {
                        var tag = pc.Tags.Where(o => o.Name == t).FirstOrDefault();
                        if (tag == null)
                        {
                            tag = new ViewModel.Tag(t);
                        }
                        q[i].Tags.Add(tag);
                    }
                    pc.Pictures.Add(q[i]);
                    try
                    {
                        await pc.SaveChangesAsync();
                    }
                    catch { }
                    dbUploadProgress.Value = (i + 1) / (double)imgcount;
                    var labelText = string.Format("{0}/{1} ({2:0.00}%)", i, imgcount, dbUploadProgress.Value);
                    dbUploadText.Text = labelText;
#if DEBUG
                    System.Diagnostics.Debug.WriteLine(labelText);
#endif
                    await Task.Delay(2);
                }
            }
            q.Clear();
            MainWindow.AddLog(string.Format("Saved {0} pictures.", imgcount));
            dbUpload.Visibility = Visibility.Collapsed;
            await Task.Delay(1500);
        }
        catch(Exception ex) { AddLog(ex.Message); if (ex.InnerException != null) AddLog(ex.InnerException.Message); }
        if (stopped)
            break;
    }
}

It usually saves around 80-100images/min, which isn't really acceptable because in most scenarios the Queuestores 2500-10000 objects. That makes it really inconvenient to use.

Am I correct in assuming that it happens because I'm saving each row separately? It also takes a while to check if the tag exists in the Tags table.

The reason why I'm doing it like this is the fact that I have to check for image tags constantly. I have to make sure that the tag isn't there already, because if there is and I add it, it throws Duplicate Exception and the image is not saved. This also means that I have to save the previous post's data before I check if tags exist, otherwise it might not detect the tags that were just added, add them multiple times and the same outcome occurs.

Is there any way to make it automatically save tags that aren't in the DB and just make it point to these that are? That would mean I can save the entire list at once, instead of sending each image separately.

Also, I'm using another context to add Sites to the DB. When I try to add some images with this task it throws an error that the Site is already in the db. For some reason it kept trying to insert the site each time I was adding the Image. Adding the .Attach() call solved this issue partially, but I'm not sure how it works. I know I'm doing something wrong because it ocassionally throws an exception and doesn't attach the site of the image. How can I optimize it?

Thank you for reading, cheers!

share|improve this question
    
Load the entire tags table at once and keep it updated in memory as you are adding images. – Steve 20 hours ago
    
With pc.Tags.ToList(), or something like that? How would that prevent the duplicates from being updated, though? – Reynevan 20 hours ago
    
You said you have to check for duplicate tags constantly, which i assume means, that you have to go to the database to check if it exists every time you add an image. I'm saying before adding images, retrieve all Tags that exist and use that to check for duplicates. This saves you a database trip every time you want to add a row. – Steve 20 hours ago
1  
Thanks. I tested that and it indeed works well when I just add all new tags to the local list instead of saving them to the database directly. It works here on my local PC, where I have ~1000 tags, but I'm not sure how this will work with the actual DB, where there's ~20k tags. – Reynevan 19 hours ago
    
Right now the increase in performance is huge :D – Reynevan 19 hours ago

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.