Questions regarding efficient storing and representing data within a software application.
1
vote
2answers
84 views
Shift bytes through a fixed sized buffer
I'm writing an InputStream that processes data streams containing "trailer" data. That is, the final n bytes of the stream is a piece of data that must be handled separately and shouldn't be returned ...
0
votes
0answers
25 views
Data-structures and memory allocation [closed]
I have a structure:
struct student
{
int id_number;
char first_name[30];
char gender;
};
When I try to enter the gender field it does not work. Skip over this step. Why does that ...
0
votes
1answer
90 views
Good Data Structure book to improve programming logic for children [closed]
My cousin (Age 10) who studies in Standard 5. He wants to learn programming and programming logic.
Which book or resources should I refer? Is there any Good Data Structure book to improve ...
-2
votes
0answers
83 views
Lectures on data structure [closed]
I am looking for good online video lectures on advanced data structures. From advanced data structure I mean topics like trees, hashing , Dynamic memory management , heaps. Any reference for good ...
-2
votes
0answers
69 views
How DateTime data type is stored in different database systems? [closed]
I'm curious about how date and time related datatypes are stored in different database architectures?? How data structures for datatime dts are different in different DBMSs?
-1
votes
1answer
54 views
Storing User Form Data
I am trying to come up with a data schema/methodology for storing the data collected from various different user application forms.
All forms contain the basic info like name, address, etc but then ...
-2
votes
2answers
165 views
About Artificial Intelligence [closed]
I am interested in starting a career in artificial intelligence.
Can anyone suggest how I could prepare for this?
What languages should I study that would be best for this career choice?
1
vote
1answer
80 views
Format text in a generic and reusable way
I would like to write some long text in some structure to allow a set of operations on that text. The question is which structure or format should I use, which suits best the use that I plan to do of ...
1
vote
3answers
295 views
The best way to store dictionary from file
I'm working on a translator in C++. Basically I want to parse the file with translations and store it in my program, so I can perform search through the words and simply access the corresponding word. ...
2
votes
1answer
70 views
Bayes filtering and data storage with expansive data sets
I am looking to write a Bayes filter that will act as an indicator of topic for a number of topics with a variable number of sources. Given a really big number of RSS feeds and here really big might ...
1
vote
0answers
82 views
Handling timeout in network application
How can I handle timeouts in a network application.
I'm implementing a provisioning system on a Linux server, the code is huge so I'm going to put the algorithm, it works as like this
Read ...
1
vote
4answers
594 views
Why many programming languages have only 2 data-structures: arrays and hashes?
Many programming languages have only those 2 structures, and even some languages that have more structures still only provide special syntax for those 2; usually, [] and {}. Why is this? Is there ...
3
votes
2answers
170 views
How should I structure a C# application that reads & writes binary data?
I have to read and write binary "chunks" of approximately 1Mb each. The data can come in the form of a stream or a in-memory byte[].
Normally I would use a Struct with a fixed layout, but there are ...
0
votes
1answer
147 views
What can Go chan do that a list cannot?
I want to know in which situation Go chan makes code much simpler than using list or queue or array that is usually available in all languages.
As it was stated by Rob Pike in one of his speeches ...
4
votes
3answers
210 views
How Immutable Sets are Manipulated
When working with an immutable set or map, like the ones found in many functional programming languages, operations that would otherwise modify the container generate a new container instead.
I know ...
6
votes
1answer
191 views
Optimal Data Structure for our own API
I'm in the early stages of writing an Emacs major mode for the Stack Exchange network; if you use Emacs regularly, this will benefit you in the end.
In order to minimize the number of calls made to ...
2
votes
3answers
128 views
Plugin Architecture: How to Handle Databases and Logging?
I am in the process of designing an architecture for a plugin based application and I have some questions about how to handle database access and logging. The goal of my application is to allow a ...
12
votes
5answers
285 views
Storing in-text metadata in a discrete data structure
I am developing an application which will need to store inline, intext metadata. What I mean by that is the following: let's say we have a long text, and we want to store some metadata connected with ...
3
votes
3answers
204 views
Is it worth to try write foolproof data structures?
The problem
We need to store data in a table-like way, but we have very strict space constraints (~1Mb per table of 10k+ rows). We store data like this:
ID | reviews | factor | score | interval | ...
4
votes
5answers
430 views
Why is the usage of string keys generally considered to be a bad idea?
This has been bugging me for a while. Most of the time, when it comes to storing data in structures such as hashtables, programmers, books and articles insist that indexing elements in said structures ...
1
vote
1answer
94 views
Storage and naming of log data
I'm in the process of rewriting a small application from console mode to a GUI mode.
The input is a file with log data from a measuring device:
...
00029;00044;00076;00044;00021
...
2
votes
1answer
63 views
How do I transparently cache data in intermediate stages of processing?
I am working with MATLAB on a model reduction algorithm. It is basically a data processing pipeline.
ckt = generate_ckt(ckt_properties);
freq = generate_fpoints(fconfig);
result = ...
2
votes
1answer
90 views
Applications with duplicate data - keep separate or combine into one set?
We currently have 3 similar, but slightly different applications that use the same data. We load the (same) data into each application that uses it. The applications are similar and use the same ...
6
votes
5answers
806 views
What is the best way to store a table in C++
I'm programming a decision tree in C++ using a slightly modified version of the C4.5 algorithm. Each node represents an attribute or a column of your data set and it has a children per possible value ...
0
votes
1answer
88 views
overworld vs on-screen 2d tile organization
I am developing a 2d game where the world is made up of tiles.
I have an overworld and a "current world":
the current world represent the current tiles on screen at any given time, currently this ...
1
vote
1answer
88 views
Database data structures for RESTful api
I am creating a RESTful API. I am struggling to decide on the best way to design my database tables around my resources.
Initially, I though a table per resource would be a good way to go, but I'm ...
1
vote
2answers
353 views
Nested Enum type in C++ or C#?
I've come across a recurring issue in a few of my recent projects in which I find myself using enums to represent state, or type, or something else, and I need to check against a few conditions. Some ...
1
vote
1answer
85 views
Marked nodes in Fibonacci heaps
I don't understand why Fibonacci heaps have marked nodes (picture).
A node is marked when its child is deleted. Quoting from Wikipedia: "[Keeping degree of each node low] is achieved by the rule that ...
2
votes
1answer
82 views
creating simple states for a stateless input
Given an input device (basically a keyboard) that reports keyup and keydown, how may I most efficiently store and retrieve information about which keys are currently depressed? My first thought was a ...
0
votes
2answers
109 views
Data structure for fast StartsWith filtering of a dictionary
I have a list of english words. I'm trying to come up with a data structure that allows me to do fast StartsWith filtering, like:
var words = dictionary.StartsWith("foo");
I was thinking of ...
8
votes
2answers
261 views
What's the difference in content between Chris Okasaki's 1996 thesis and 1999 book, Purely Functional Data Structures?
I want to read Purely Functional Data Structures. I've easily found the thesis (which is freely available as a PDF), but see that there's a book available also. So I'd like to know what the ...
-2
votes
1answer
98 views
Non-obvious use of graph [closed]
What are the uses of graphs that can be considered non-obvious. For many graph examples, it is very clear for people who knows even a little about graph that it can be represented as a graph, like ...
5
votes
4answers
479 views
What is the point of using lists over vectors, in C++?
I've run 3 different experiments involving C++ lists and vectors.
Those with vectors proved more efficient, even when a lot of insertions in the middle were involved.
Hence the question: in which ...
7
votes
2answers
281 views
Using a stream manipulator (endl) or a newline escape character (\n)?
I don't have a specific context in which I'm asking the question, but while I was reading a beginner book on C++ I noticed the use of both an endl stream manipulator and a newline escape character ...
0
votes
2answers
255 views
Algorithms for data structures in distributed system
The hash table data structure can be easily spread across multiple machines with a simple algorithm to distribute the keys:
machine_to_query = item_key % machine_count
When you want to read and ...
5
votes
1answer
338 views
How do scalable bloom filters work?
I was reading up on scalable bloom filters and could not understand how each time a constituent bloom filters fills up, a new bloom filter with larger size is added.
The elements that contributed to ...
2
votes
2answers
381 views
How to increase the efficiency of an Immutable Queue in java?
I have been developing an Immutable queue in java, but the version I have developed is supposedly slow and I have been suggested to create a faster version of the same and I have no idea as to how to ...
2
votes
2answers
248 views
Thoughts on a custom data structure for a node/chain implementation
I am designing a simple GUI that will allow a user to create a GPX route by clicking repeatedly on a map panel. I am faced with a design dilemma for how to represent the nodes and route. This ...
4
votes
2answers
351 views
Creating huge decision tree
I'm to write an event correlator. A fundamental part of the system will be a decision tree that recognizes the origin of the fault basing on recorded states and log files.
Often many accidents will ...
3
votes
6answers
278 views
Should a database table(s) structure match its intended data structure(s) in the logic?
This question branches out of this question, What are the differences between algorithms using data structures and algorithms using databases?.
The General Question
Should a database table(s) ...
5
votes
4answers
125 views
Representing a rule in a ruleset
How to represent rules for a rule engine as objects?
A rule would be
if (booleanExpression(input)) then a chain of generic actions" else next rule
...where the generic actions might be e.g. ...
7
votes
4answers
614 views
What are the differences between algorithms using data structures and algorithms using databases?
The General Question
What are the differences between algorithms using data structures and algorithms using databases?
Some Context
This is a question that has been bugging me for some time, and I ...
1
vote
3answers
365 views
What data structure could a word processor use to map the user's caret position/text selection to its internal representation of the document?
Word processors (e.g. Microsoft Word) display documents to the user as styled text. The user can select a part of this text and apply styles to it, and edit the text.
The word processor must (I ...
3
votes
1answer
88 views
Representing domain objects
This is related to my recent question regarding naming awkward domain objects. A number of answers indicated I was using the wrong representation for the domain objects. To summarize, I chose to use ...
0
votes
1answer
329 views
Simple dependency tree diagram generator
I have a need to produce a simple dependency tree diagram. The input data would be in the following simple format:
ITEM_NAME DEPENDENCY
----------------------------
ITEM_101 ITEM_75
ITEM_102 ...
3
votes
2answers
569 views
How, when, and why do developers use custom data structures in python?
Python's lack of pointers makes data-structure construction intuitively more challenging, and has so much built in functionality that, as an amateur, I can't see when, why, or how you would create ...
20
votes
3answers
710 views
How do I express subtle relationships in my data?
"A" is related to "B" and "C". How do I show that "B" and "C" might, by this context, be related as well?
Example:
Here are a few headlines about a recent Broadway play:
David Mamet's Glengarry ...
3
votes
3answers
369 views
Is it conceivable to have millions of lists of data in memory in Python?
I have over the last 30 days been developing a Python application that utilizes a MySQL database of information (specifically about Norwegian addresses) to perform address validation and correction. ...
2
votes
1answer
746 views
Sets Data Structure in Golang
I really like google golang but could some one explain what the rationale is for the implementors having left out a basic data structure such as sets from the standard library?
-3
votes
1answer
355 views
map data structure in pacman [closed]
I am trying to make a pacman game in c#. I have done some basic work and have previously replicated games like copter-it and minesweeper. I am confused about how to implement the map in pacman. Which ...
