current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 0 down vote favorite

I am working on an application that reads a log file, looks for a specific string, sends alert emails and saves data from the file into a database. The reading functionality is working well, but I am coming across an issue when I stop and restart my program. Basically, it always starts from the beginning of the log file, so I am rereading information I have already read. Please reference the example below.

Consider I have the log file with the following lines

1 0708 1200 Error in log
2 0708 1230 Received invoice 00001
3 0708 1231 Received invoice 00002
4 0708 0130 Error in log 
5 0708 0135 Received invoice 00003
6 0708 0200 Received invoice 00004
7 0708 0230 Received invoice 00005
8 0708 0235 Error in log

In this example, say I ran the application. The application would start reading at line 1 and continue until I stop the application (picking up any lines that are added to the file as well). Let's say I stop the application when I get to line 5. If this were the case, I would have received an alert e-mail from line 1 and line 4. I would have also saved lines 2 and 3 in the database.

My problem/question comes from when I want to restart the application again. Currently, the application will start reading at line 1 again. This is not an issue when it comes to saving data because I have placed a clause to not save if the line has already been added. However, when it comes to sending e-mails I will receive an e-mail for lines 1 and 4 again.

After all of the that rambling, I will present my actual question / inquery. Is there a way to stop a program (it completely stops the process) and begin reading from the spot in which I stopped reading from? I cannot think of a way to save the place holder for the file and use it once I start the program again.

I have considered saving the file pointer value to a text file and reading from there at every start up, but I was wondering if there was another way or any suggestions to improve this process. I know this question is abstract, but I am reaching out for extra brains on this issue.

Little information:

  • This is a java program
  • This is a web app running on a tomcat server
  • The server that tomcat is running on is a unix server
  • This is a constantly running application that should only be shut off during an issue or change
  • The logs being scanned are very large with 500,000+ lines
  • The logs turn around (get replaced with empty log) once they reach a certain size. This means that when I save the file pointer, I have to make sure I am in the same file before I start reading again, otherwise I want to start from the beginning

Any help / suggestions would be very helpful. Sorry for the wordy question, please let me know if anything is unclear. Thanks!

share|improve this question

2 Answers 2

up vote 1 down vote

If I were you I would simply keep a place holder in a separate file.

If you are looking for alternatives though, you could place a special character directly in the log. Like a flag/number as the first line of the log that would update after reading every line. If you are working with several files, you could check each file for that first line flag or modify the file name to denote whether or not it is currently in use.

share|improve this answer
    
Yes a place holder in a separate file is what I was thinking. I was also considering storing in a database... I will be running 3 instances of this program at a time (reading 3 different logs) so this way I can pull for the specific sever being tracked –  user1998581 Jul 8 at 18:37
up vote 0 down vote

One of the ways I can think is to store the line number of the previous read and then start from that position, something like this and keep reading until eof.

public static void main(String[] args) throws IOException {
BufferedReader in = new BufferedReader (new FileReader("Profiles.ini"));

String info = "";
int startLine = 17;
int endLine = 22; //EOF in your case

for (int i = 0; i < startLine; i++) { info = in.readLine(); }
   for (int i = startLine; i < endLine + 1; i++) {
       info = in.readLine();
       System.out.println(info);
   }

   in.close();
}

I got this extract from here.

OR

Take a look at this thread : http://stackoverflow.com/questions/2312756/in-java-how-to-read-from-a-file-a-specific-line-given-the-line-number

share|improve this answer
    
I agree with your logic...when I start back up I will jump to the file pointer that I was previously on. My problem comes with saving that file pointer value while closing the process of the program. I am trying to think of a way that once the program/process is killed, to start it back up and resume at the exact spot. –  user1998581 Jul 8 at 18:30
    
That still requires to read the whole file. It would be better to save the absolute position in the file with file.getFilePointer() and later, when you restart your program, position the file to that position with file.seek(position). –  Florian F Aug 22 at 8:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.