Now to tackle the unknown.....

On our project work list we have this one task, that is about as clear as mud. So I decided I will at least look into it. It looks like the web service I was working on a few days ago has a few parts to it that all run as seperate threads. For anyone that doesn't know and wants to, a thread is kind of like multi tasking. For those that use windows, you can think of this like the running processes in task manager. While a process and a thread are not the same, they are close enough for this discussion. Ok, so this task involves on thread that looks at the database every 15 minutes and decides if it needs to send out some emails. The problem is that under heavy load this thread can generate an error. When this happens it sits in this error state for the next 15 minutes. The IT department uses a program called Nagios to check that everything is ok. Right now if this thread does die it reports an error in Nagios and the IT department is notified, even though it was just an issue with increased load. My plan on to setup a loop. We would try to run the thread, if it failed we will wait 30 seconds, then try again. We would repeat this until either 3 minutes has passed, or the thread runs with no errors. If 3 minutes passes and the thread still doesn't run successfully then we go into an error state and let Nagios report the error to IT. I am not sure if my project manager wants me to work on it or not, but I emailed him what I have found out and what I planned and I will let him decide!