per-process bandwidth accounting
Dear Lazyweb,
I want to track bandwidth used by processes on my colo server on a per-process basis. I'm running Apache, BitTorrent, Freenet, and miscellaneous other things like automated backups. I have a monthly bandwidth cap of 1800 GB. I'm running FreeBSD 5.3. What are my options?
I want to track bandwidth used by processes on my colo server on a per-process basis. I'm running Apache, BitTorrent, Freenet, and miscellaneous other things like automated backups. I have a monthly bandwidth cap of 1800 GB. I'm running FreeBSD 5.3. What are my options?
Have a donut and coffee, and pay somebody else to do it? =oD
heh..sorry, that's all I've got. =oþ
heh..sorry, that's all I've got. =oþ
![[info]](http://stat.livejournal.com/img/userinfo.gif){kind=small}
decibel45
2005-05-04T20:08:53Z
I'll add greg to the list of people who will be helping to toss your butt out of a plane this summer. (Something he's already done.)
lucky_mcgee
2005-05-05T08:19:58Z
OH NO!! There are TWO of you?!?! =oO
I'm SURE to be killed.
I didn't realize you guys were "real life" type friends. You guys should post more so I can figure all this stuff out.
At the rate either of you post, and as much content as you put into your posts, I won't know your favorite food, drink or color until...oooh....maybe 2007...? ;oP
I'm SURE to be killed.
I didn't realize you guys were "real life" type friends. You guys should post more so I can figure all this stuff out.
At the rate either of you post, and as much content as you put into your posts, I won't know your favorite food, drink or color until...oooh....maybe 2007...? ;oP
ghewgill
2005-05-05T13:43:12Z
Well I first encountered your journal when I was over at
snaxxx
's place with
decibel45
and
_fool
, and we were planning SXSW 2005 Nuclear Taco Night. I can't find it now, but you invited people to call your voice mail and leave a message. We did, collectively, and I recall
snaxxx
mentioning that it made an impression on you. :)
You definitely should plan a trip to Austin sometime so we can throw you out of an airplane, feed you , have some homebrew (and make some more), and so on. And a new thing - last night we made fresh mozzarella cheese (first time for all of us) that was absolutely to die for.
snaxxx
's place with
decibel45
and
_fool
, and we were planning SXSW 2005 Nuclear Taco Night. I can't find it now, but you invited people to call your voice mail and leave a message. We did, collectively, and I recall
snaxxx
mentioning that it made an impression on you. :)
You definitely should plan a trip to Austin sometime so we can throw you out of an airplane, feed you , have some homebrew (and make some more), and so on. And a new thing - last night we made fresh mozzarella cheese (first time for all of us) that was absolutely to die for.
lucky_mcgee
2005-05-05T18:50:44Z
Ooooh YOU were the Cartman??? xoD hahaha! That was great. My favorite message of the night. And it's still on my machine. I refuse to erase it.
I'll be making my annual summer road trip in a few months, and am actually in the process of planning out my locations I'll be going to right now, and Austin is already on the list!
But I can't make any promises about jumping out of planes. heh. However, I'm more than game for tacos drinking and all the cheeziness you people have to offer! =oD
I'll be making my annual summer road trip in a few months, and am actually in the process of planning out my locations I'll be going to right now, and Austin is already on the list!
But I can't make any promises about jumping out of planes. heh. However, I'm more than game for tacos drinking and all the cheeziness you people have to offer! =oD
ghewgill
2005-05-05T22:02:37Z
decibel45
is the loud one, he was cartman.
I don't think we've tried drinking the tacos before, though. You'll have to let us know how that goes.
lucky_mcgee
: 'scuse me for forgetting a comma.2005-05-05T22:05:29Z
tacos, drinking...
Better!? =oP
Better!? =oP
decibel45
2005-05-05T21:57:14Z
lucky_mcgee
2005-05-05T21:59:50Z
OH GOD NO. The pressure.
Can't we just eat nuclear tacos instead?!? =oD
Can't we just eat nuclear tacos instead?!? =oD
ghewgill
2005-05-05T22:09:24Z
Don't worry, the pressure isn't a problem. At roughly 1 inch of mercury per thousand feet, the pressure at exit altitude is just over half what it is on the ground. :)
lucky_mcgee
2005-05-05T22:12:52Z
You're not helping the fears here.
decibel45
2005-05-05T22:17:46Z
Sure, after we get back on the ground. :)
lucky_mcgee
2005-05-05T22:28:42Z
I can't believe for like 0.009 seconds, after I read that, I actually entertained the idea.
Thank the good lord for reality. heh.
Thank the good lord for reality. heh.
ghewgill
2005-05-05T07:16:16Z
As you will no doubt see as this project proceeds, that is so not my style. :)
lucky_mcgee
: So I'm not one of the cool kids...2005-05-05T08:17:01Z
heh...I'm sure it isn't, but like I said, it's the best I've got. :oþ
paradox0220
2005-05-04T18:51:06Z
That is a cool idea. Track and limit bandwidth per process (or process type, all httpd clumped together in stats and control). Time for greggie to do some kernel hacking? :)
ghewgill
2005-05-05T07:17:16Z
I don't think it's going to need kernel hacking. I'm pretty sure I can do this in all user mode code.
decibel45
2005-05-04T20:08:10Z
The only thought that comes to mind (and this is a bit ugly, so hold on to your hat) is this: NAT let you forward ports from the outside interface to a specific IP behind NAT. So, if you can collect SNMP data on aliased IP addresses, you can run NAT and have it point at a bunch of aliased IP's.
Another possibility is afaik ipfw will let you filter by user, so you might be able to do something with that.
Maybe your best bet would be to write some code that ties into tcpdump and tracks how much data went where.
Another possibility is afaik ipfw will let you filter by user, so you might be able to do something with that.
Maybe your best bet would be to write some code that ties into tcpdump and tracks how much data went where.
ghewgill
2005-05-05T07:20:52Z
Well see I've got this great colo server that's not behind nat or anything, it's plugged right into the internet. It would be a real shame to set up some kind of virtual nat layer on that machine!
I do run the major services on my machine under separate accounts, so perhaps I could look at the per-user accounting angle for this. Maybe a jail with the same root directory.
Using data from bpf (berkeley packet filter, which tcpdump uses on freebsd) wouldn't tell me where the data comes from, except by port number which is annoying for dynamic services like bittorrent and freenet.
I do run the major services on my machine under separate accounts, so perhaps I could look at the per-user accounting angle for this. Maybe a jail with the same root directory.
Using data from bpf (berkeley packet filter, which tcpdump uses on freebsd) wouldn't tell me where the data comes from, except by port number which is annoying for dynamic services like bittorrent and freenet.
decibel45
2005-05-05T14:39:21Z
I'm pretty sure jail will want you to setup seperate IPs for the jails.
If you use the per-user/group filtering in ipfw, you could then use tee to send it to a divert port, which could be a daemon that does nothing but count bytes (and packets). It might not be 100% accurate compared to SNMP because it would ignore lower layer overhead.
If you use the per-user/group filtering in ipfw, you could then use tee to send it to a divert port, which could be a daemon that does nothing but count bytes (and packets). It might not be 100% accurate compared to SNMP because it would ignore lower layer overhead.
2005-05-04T18:20:20Z