Major Memory leak (Mapped File memory) during SageTV recordings

[bradvido]

When SageTV (v7) records a file, it seems to map the file to memory. However, it never cleans up after itself and keeps consuming more and more memory. Using RAMMap (Sysinternals), I can see that SageTV consumed 4+GB of RAM while recording the news this morning. As soon as the news stopped recording, 4GB of RAM freed up on my server.

Has anyone else noticed this?

Refer to these screenshots:

RAMMap showing the Mapped File memory usage when recording the news.


When the recoding ended at 9:00 AM, the memory usage instantly cleared out:

[gplasky]

I don't know how that can be happening because I only have 2 GB of RAM and make multiple recordings at the same time from 4 GB to 8 GB or more in size. Never saw this.

Gerry

[bradvido]

Maybe Sage is writing to memory (i have 8gb avail) instead of deleting an old recording from the disk and making space?

I don't know but it needs to be fixed

[GKusnick]

I'm confused. You called it a leak and said it never gets cleaned up. But then you ended by saying it does give the memory back when the recording is finished. If that's true, then it's not a leak; it's just a large temporary memory demand.

I'm also not clear on whether the numbers you posted refer to actual physical memory in use, or just address space allocated. Mapping a 4GB file will take up 4GB of address space; that's what file mapping means. But it doesn't necessarily mean that all four GB will be swapped into physical memory at the same time.

Is there some performance issue you're trying to diagnose? Or did you just see these numbers and think something must be wrong?

[bradvido]

Originally Posted by GKusnick

I'm confused. You called it a leak and said it never gets cleaned up. But then you ended by saying it does give the memory back when the recording is finished. If that's true, then it's not a leak; it's just a large temporary memory demand.

I'm also not clear on whether the numbers you posted refer to actual physical memory in use, or just address space allocated. Mapping a 4GB file will take up 4GB of address space; that's what file mapping means. But it doesn't necessarily mean that all four GB will be swapped into physical memory at the same time.

Is there some performance issue you're trying to diagnose? Or did you just see these numbers and think something must be wrong?

Yes, it was a bit of a misnomer calling it a leak, more of just a hog, since it does get cleaned up when the recording is done.

Noticable, large performance issues are caused by this which leads me to believe that the memory space is reserved for Sage and not shared in anyway.

I'm just curious as to why Sage is allocating that much space? Is it some sort of buffer? Write recordings to disk and let the OS handle the buffer.

[paulbeers]

I am not sure this is true at all. Like gplasky, my server only has 4GB of physical memory and I record several recordings at once of 6+GB per recording. Based on what you are saying, I would need to have something like 25GB worth of memory! Now could Sage be dumping all the streams onto the hard drive which is being misconstrued as using virtual memory and then after the steam is completed and properly stored, the system no longer sees it as virtual memory but rather a file storage? This would make a lot more sense.

There is no way Sage could function on my server if the recordings were being stored in RAM. All of my clients would completely suffer....

[gplasky]

I would think the Windows swap file would come into play here.

Gerry

[nyplayer]

I have never seen this and I only have 3 gigs of memory on my server and record at times 7 recordings simultaneously and still have plenty of memory left to do comskip and run videoredo.

[bradvido]

Originally Posted by paulbeers

I am not sure this is true at all. Like gplasky, my server only has 4GB of physical memory and I record several recordings at once of 6+GB per recording. Based on what you are saying, I would need to have something like 25GB worth of memory! Now could Sage be dumping all the streams onto the hard drive which is being misconstrued as using virtual memory and then after the steam is completed and properly stored, the system no longer sees it as virtual memory but rather a file storage? This would make a lot more sense.

There is no way Sage could function on my server if the recordings were being stored in RAM. All of my clients would completely suffer....

I'm not saying Sage needs or requires physical RAM for each recording. I'm just saying it uses my memory because i have a lot to spare. I would like it to instead write to disk (like it does on your computer because it can't reserve that much memory).

[panteragstk]

Something else is at work here. Nobody else seems to have this issue. If you can grab a screen from process explorer so we can see what else is going on.

[Fuzzy]

This is as mentioned, the memory mapping. It is not using that much physical memory, it's just the way the system writes the stream to disk. Sage can only use as much memory as the JVM can hold, which caps out around 1.2GB. Win7-64 can map up to 8TB of address space. This can be used for everything from actual RAM, paged storage, mapped file access, or device driver memory maps. Sage, rightly so, allocates the recording file as a mapped file, so all it has to worry about is writing to the mapped location, and the windows system deals with actually writing the bits to disc as it sees fit. (this is why process explorers and such will show the drive access as coming from System, and not from SageTVService).

See: http://en.wikipedia.org/wiki/Memory-mapped_file

[bradvido]

Ok that makes sense. But what if another process wants to use that memor space? From what Ive seen, sage doesnt share

[Fuzzy]

Applications don't ask for specific memory addresses. Applications ask for a certain sized chunk of memory. Windows then tells it ok, you can have this range of address space. That doens't even mean that space is physical memory. Even if you only have 1GB or ram in your system, windows has 8TB of available address space it can use for user mode programs. Addresses are not necessarily places in RAM, they are an index to data. That data can point to physical memory, but it can also point to files, devices, whatever.

[GKusnick]

Originally Posted by gplasky

I would think the Windows swap file would come into play here.

Not for mapped files. VM pages allocated to mapped files are backed by the files themselves. The page file comes into play only for pages that aren't backed by anything else (Java heap for instance).

[OldMariner]

Bradvido, SageTV is also hogging memory on my system, and this is a new problem. It eventually brings my W7-64bit system with 8GM RAM to it's knees, not due to CPU usage, but due to using all the RAM. SageTV gradually uses more and more memory while recording. When the recording is complete it releases this memory. But in the meantime, playback is slow to respond, and eventually nearly locks up when all available memory is used. Then all applications, such as the browser and Outlook also become slugs when the system has to go to virtual memory. Shutting down the other applications seems to have no effect as SageTV gobbles up that memory also.

It just started doing this. I am wondering if it is due to recent JAVA 64 bit version 6 update 29 that I installed. I have both the 32 bit and the 64 bit JAVA's. I will shut down the SageTV service and uninstall the 64 bit JAVA and see if that fixes the problem. Maybe I will also uninstall the 32 bit JAVA and reinstall it. This is one recent change that I can think of that might be doing this. I also recently installed the STV update; 2011110601. Sure hope that was not the cause.

Hope this info help to lead to a solution.

I will post further info as soon as I can.

[zoop]

several people in other threads are reporting issues with java 2.29... updating java should be looked upon as strictly optional, especially on a server that's reliant on it

/jer

[Fuzzy]

Also, installing 64-bit java should have no bearing on sagetv, as it is a 32-bit java program. It won't use the 64-bit JVM, no matter what you install.

[davenlr]

I have 4gb of ram on a win7 64 bit server. Sage is the only service running (other than standard windows services). I disabled the swap file, and rebooted. Computer reported over half the physical memory free. After about 5 days of running, the computer stops with an out of memory error.

I have re-enabled the swap file, but thought it was interesting that it runs ok for 4 days, and then runs out of memory. Never ran any programs to see what was consuming it though. Its either windows or sage or Java.

[Fuzzy]

Well, I can pretty much guarantee it isn't sage directly, as it can't use anything outside of the limits of the JVM (which will cap out at about 1.4GB max).

[briands]

My new Win7 x64 install is having a similar problem. The task manager also shows a process javaw.exe as growing. This is with Java 1_0_29.

I have not increased the heap on this system, so that is one of the next things to do along with researching the current "best" java version.