Update Time info

Suman requested that I look at the update time closer, here's a graph of the amount of time the update calculation took versus when during the trace the update was run. This is a normal file with no delay added to the calculation.

There seem to be a couple spikes during the trace where the calculation takes about 5.5 seconds, these occur mostly in the warm up period and then infrequently afterwards. Only 58 of the roughly 2400 of the calculation taking longer than 1 second to compute.

The warmup period in this trace takes approximately 1,900 seconds.

Results with another trace

I ran the simulation on another trace file with delays added to simulate slower processors. I did basically the same tests as before, the main difference with this run was I used slightly more varied delays with a trace of a large delay of 1 second. Here are the results:

Hit Ratio
This is the delay from each period, as you can see again, most of them are fairly close together towards the end. The exception being the trace file with 1 second delay, this one did considerably worse for a hit ratio than the others. Also we see again the areas where the hit ratio drops drastically in the traces with no delay and the trace with 10ms delay. I think these must be areas with a lot of packets that don't fit the rules in the cache and therefore generate a cache miss.

Cumulative Hit RatioAgain the results are fairly similar compared to the other trace. Two interesting things here, the 10ms delay simulation performed better than the simulation with no delay. The other thing to notice is the 1 second delay simulation is again much lower than the rest of the traces.

Update Time
Here we see why the simulation with 1 second delay performed worse than the rest of the simulations. It looks like the delay was large enough that it made the calculation of the new ruleset much more difficult, the update delay doesn't even get below 3 seconds in this case.

update

I'm running a similar test as in the previous post with another trace file to compare the results. The simulations take a while to run but I should have enough traces by the end of the week.

Hit Ratio vs Cache Update Time

I ran the simulator with a tracefile and added delay to the rule cache update method to simulate this running on a slower system, I used the trace5a trace file and ran 5 different simulations: no update delay, 1 millisecond update delay, 30 millisecond delay, 60 millisecond delay, and 100 millisecond delay.

The results are as follows:

Hit Ratio
The hit ratios for each individual update period varied during the start but they all converged to around 98% towards the end. I'm not quite sure about the spikes downward in the hit ratio graph, there were likely just periods where there were a lot of cache misses in this trace. This could be verified by looking at the results of another trace file.

Cumulative Hit Ratio

The overall averaged hit ratio was better with less delay but this could likely be due to the variation between the hit ratios in the start of the simulation.


Update TimesHere is just a graph of the average update time for the different simulations, there doesn't seem to be as distinct between the time delays as I would have liked to see but there seems to be a general distinction between the no delay simulation, the mild delay simulations, and the 100 millisecond delay simulations. I am not sure why the other ones are so close but it might have to do with the rules being calculated over different packet intervals.

Overall it seems that the update time most effects the hit ratio while the cache starts up and the averaged hit ratio but the hit ratio for individual periods seems to be fairly consistent with delay.

Traces

I'm running some traces on the simulator with different delay values and once they finish I will be able to graph the output and we should see what the correlation is between the cache update time and the hit ratio.

New code version

I talked with Dheeraj and he said that in the new version of the rule cache code that he sent to David would take care of the timing issue that we were seeing before. He also mentioned that this new version didn't use the large trace files so it will not have to do file I/O and should run faster in the rule cache update.

I am going to run the simulation a couple times and put a delay in the update rule cache method to see how the hit ratio differs with different delays in the calculation of the new rule set.

Change the code?

From a brief meeting with Suman today, we might have to make a slight change in the code to account for the problem I mentioned in the previous post as far as the hit ratio not being effected by the cache update time.

I took a look at the code for the rule cache simulator. I think the change will have to be with how the simulator determines when the next cache update will occur. I am meeting with Dheeraj later this week and I will ask him about this.

I found some code that Dheeraj had commented out that looks like it will do what we want for the change in the code, I'm going to uncomment it and give it a try to see if this solves the problem.