Blade servers
Links: home · search · speed test · login · more ·
Links: Reply New Topic
Forums » Canadian » TekSavvy » Blade servers
page: 1 · 2 · 3 · 4 ...18 · 19 · 20
milnoc @ 7th Jun 06:35PM:
Blade servers
All right! Uber Geek Alert! In other words, I don't know where else to ask this question. :)
As some of you know, I'm working on launching a cable television channel later this year. The automated broadcast system for this channel will be entirely computer based, and the distribution will be for the most part IP based. The performance requirements for the system will not be too high, but I'll still need quite a bit of equipment, and at least 10 terabytes of hard drive space for the video files.
To make life easier, I'm considering getting a blade server package which will be installed at a datacenter. I figured I can squeeze everything I'll need for the next two years in a single IBM Bladecenter S chassis. Throw in a keyboard/monitor drawer, and I can fit everything in a 1/4 rack with 2U to spare.
The question is: are IBM Bladecenters reliable and cost effective these days? I'm having a lot of trouble finding ANY reviews on the product, including credible complaints!
I'm also checking out the offerings by HP and Dell, but haven't figured out if everything I'll need for the next two years will fit in that same 1/4 rack. If I have to upgrade to a 1/2 rack and extra power, the monthly co-location costs will double, possibly negating any potential cost savings achieved by going with a cheaper equipment provider (the equipment will be leased, not purchased).
So any opinions out there? Has anyone here worked with IBM Bladecenters or know someone who has worked with them? Was the experience on the whole positive or negative? And how was the cost, set-up and long term maintenance? Good or bad?
Thanks!
reply
twizlar @ 7th Jun 07:04PM:
Re: Blade servers
Beyond the cost and expandability issues down the road, the datacenters that WILL let you run that high of power density will be well out of your price range for a 1/4 rack. Fully loaded you are talking a power density of ~120-130 amps(usable) per rack, which isn't going to happen in most datacenters.
You would be better off to start small with a basic storage server with a couple front end units and upgrade as its needed from there. As far as IBM goes, they make good equipment, personally I've always found their stuff overpriced compared to most of the other major players like HP, Dell, Supermicro etc.
--
Broadline Networks Inc.
reply
HeadSpinning @ 7th Jun 07:33PM:
Re: Blade servers
When we built our data centre, we designed for around 5kw average dissipation per cabinet, or around 300 watts per square foot. We will let customers go over that power density, but will charge accordingly for power (and implictily, cooling).
You'll find that some data centres might not allow you to put higher power density in, so the money spent on blade servers might not be justified.
Find out the costs of the equipment, the savings in space, and the cost of power at the data centre you select, and do the math before you make a decision.
reply
milnoc @ 7th Jun 08:34PM:
Re: Blade servers
The datacenter I'm considering charges $100/mo for the 1/4 rack, and $250/mo for 120V/15A power. The blade configuration I'm considering will have a maximum capacity of 1900/2900 watts depending on the voltage (I don't know which is 120 volts and which is 240 volts). The chassis itself will start off with two blades each with a single 2.5 GHz Xeon quad-core processor.
The preliminary monthly costs are as follows:
1/4 rack with 120V/15A UPS/generator backed-up power: $350
Bladecenter lease: $750 (approx.)
50 Mbps unlimited bandwidth: $350
Total: $1450 per month.
If I get a system that takes up more room and consumes more power, the extra 1/4 rack and power will cost me an extra $350 per month. If the monthly price difference between the IBM blade system and a competitor's blade system is less than $350 for similar specs, then I might as well stick with the IBM system.
But how reliable is the IBM system? If historically it's had many more problems than its competitors, then the monthly cost savings are totally meaningless.
reply
nitzguy @ 7th Jun 09:01PM:
Re: Blade servers
Rock Solid Milnoc. If it has IBM on it, you can count on it. Since our company picked up doing server support for the company I work for now, large mining outfit, I've had to go to the Core room for our IBM equipment 0 times. HP Server...had to reboot it manually about a half dozen times...stupid netapp....garbage I don't even want to talk about how that thing wakes me up in an inconvenient manner.
We have some xSeries equpiment that runs our print servers, and some other stuff that I can't remember off the top of my head. But, for the most part, IBM Server = ultra reliable, even if its a little more pricey than its competitors. If you're getting it at a good price, then I say go for it.
But that's just my opinion :).
reply
NeTwOrKDawg @ 7th Jun 09:24PM:
Re: Blade servers
I have a client which had a full rack of IBM XSeries machines colo'd at Telus. Out of 7 servers, every one had the RAID controller changed at least 4 times. EACH.
Since migrating over to HP DL380's... ZERO raid controller issues. And the iLO on the systems gives great remote console access.
Even my employer refuses to use an IBM servers in our own datacenter. And we are a very large IBM partner to boot. But we only use HP in the datacenter. Why? Because years of experience of IBM vs. HP, HP proves more reliable with less parts replacements.
reply
CanerisErik @ 7th Jun 09:30PM:
Re: Blade servers
said by NeTwOrKDawg :
Since migrating over to HP DL380's... ZERO raid controller issues. And the iLO on the systems gives great remote console access.
...
But we only use HP in the datacenter.
Same here. Our racks are filled with HP ProLiants (and ProCurve switches, but that's off-topic). No issues yet; they are rock solid hardware, we're able to hot-swap most components and iLO has eliminated KVM cables entirely.
The dedicated servers that we let our customers use are all HP ProLiants, but we also eat our own dogfood and use them for internal infrastructure.
--
Erik - Caneris - Internet solutions and more.
reply
milnoc @ 7th Jun 09:34PM:
Re: Blade servers
That reminds me. I have to check my previous employer's old IBM Netfinity 5500 server. It's still used as a Linux server for their Mantis system, but one of the drives in the RAID array may have died, and there's no more hot swap. (they stopped fixing it. It has a 500 MHz P3 for crying out loud! Break down already!)
No worries. The database is backed up daily to the Windows server, which is backed up daily on the external backup device (external 2.5" hard drives).
As for the IBM system I'm currently configuring, everything will be redundant including power supplies, RAID controller, and network interface. Even the two blades will be clones of each other so that if one blade breaks down, the other blade will keep on working.
BTW, the datacenter I'm considering has already dealt with IBM blades, so no problems there.
I'll check out the ProLiant series now.
reply
twizlar @ 7th Jun 09:50PM:
Re: Blade servers
Going to have to agree about HP, thats all we use for our own services and dedicated servers for customers. They have been rock solid across the board.
--
Broadline Networks Inc.
reply
jfmezei @ 7th Jun 10:37PM:
Re: Blade servers
If you have a blade chassis with only 2 blades in it, you are probably spending a lot of money for the blade itself and colo costs.
If you really know that you will needto fully populate the blade cabinet "quickly", then yes, you can start with a sparely populated cabinet and fill it out. But if this will happen over a period of years, you can to get some some of commitment from the vendor that "years from now", the boards for additional CPUs on that model of blades will still be available.
Don't trust what HP says unless it is backed by a contract. They change products at a whim.
You also need to look at total network throughput on the blade cabinet. If all blades communicate through the same backbone to reach their respective ethernet cards, you may encounter some congestion in the cabinet. Since video is bandwidth intensive, you want to make sure you can grow ethernet bandwidth linearly with each added blade and not hit some limit before your blade cabinet is fully populated.
reply
lcdguy @ 7th Jun 11:09PM:
Re: Blade servers
You should also consider HP's offerings. we use ton's of HP hardware at work and it's all rock solid. The other reason is i believe both HP and IBM have blade solutions that can run off of standard power and not just 3 phase. We use ours for visualization and are running around 20-30 servers of 3 blades. just my 2 cents.
reply
milnoc @ 7th Jun 11:11PM:
Re: Blade servers
jf, I'll definitely check out the bandwidth of the internal network. This shouldn't be a problem for the video files, but it may become a problem one day for the Internet access.
The reason for the blade system is simple. Because I have to retain about three months of programming material, and to properly host all the BitTorrent files my channel will generate, I'll need between 10 to 20 terabytes of storage on a single RAID array, accessible to all of the servers. I also need a solution where I can easily add a server whenever the increased popularity of my channel warrants it, and where every single component has a redundant backup component in case the main component goes down, allowing my channel to remain on the air without interruption. A blade system satisfies all of these needs perfectly.
The RAID array will be split in two. The first array will contain eight 2 TB SATA drives, and the second array will contain four 1 TB SAS drives. Both arrays will each have one parity drive and one hot swap drive, or two parity drives if the blade system can support RAID-6 P+Q. The slower SATA drives will hold all the never-updated-always-read video files, while the faster SAS drives will hold all the I/O intensive files such as the automated broadcast system's master database.
Do note that the blade system will be used mainly for data storage as well as IP based program distribution to the smaller cable companies. For the live broadcasts via dedicated fibre, that will most likely be done on a separate external server which will be installed only when I can justify the high cost of the dedicated fibre connection, or if ever the larger cable companies agree to cover the costs.
reply
twizlar @ 7th Jun 11:15PM:
Re: Blade servers
Im not sure why you would want to use a blade center for such specific and generic tasks. You can slap a 10TB box together for much less than the cost of the bladecenter and drives.
--
Broadline Networks Inc.
reply
jfmezei @ 7th Jun 11:28PM:
Re: Blade servers
Your storage system can connect via fibre to many different hosts. Not really different than many hosts on a single blade.
You pay a premium for a blade. It starts to pay off only once the blade is fully loaded.
Also, with a blade, you need to consider the hot swappable capabilities. You don't want to have to power off the whole array to add/remove a faulty component (blade in front, network and other cards in the back).
I think the HP product has dual power and ventilation, but not sure about hot swap capabilities.
Blades generally do provide you with better systen management capabilities over separate systems (especially for non windows systems that don't require a VGA console). The HP one (and I assume the IBM one too) have a management console that lets you connect to any of the systems in the blade from a single location.
However, blades and generally 1U systems don't give you the latest and greatest in terms of CPU performance due to space/heat consideration.
In terms of remaining on air no matter what, you need to consider what sort of hardware/sofware combination will give you that, especially if you want to do load balancing during normal times so all yoru resources are used to provide the service.
reply
CanerisErik @ 7th Jun 11:33PM:
Re: Blade servers
said by milnoc :
Do note that the blade system will be used mainly for data storage as well as IP based program distribution to the smaller cable companies. For the live broadcasts via dedicated fibre, that will most likely be done on a separate external server which will be installed only when I can justify the high cost of the dedicated fibre connection, or if ever the larger cable companies agree to cover the costs.
Too bad you're not at 151 Front, this "high cost of the dedicated fibre connection" would have been peanuts...many of the large cable companies are here, including Rogers, Videotron, Shaw, and others, so you'd be able to connect to their networks easily.
--
Erik - Caneris - Internet solutions and more.
reply
Stormlord @ 7th Jun 11:34PM:
Re: Blade servers
Hi, First the best way to go is take san for your data put it in iscsi , for hardware , personnely I use IBM Xseries Server but I also work with Hp and Dell and no problem , may be dell is little cheeper. Take san with 2 or 4 giaga port and switch dedicated for you san. Use Vmware Vsphere on 2u servers for beginning , you can add more server when load is need.
Good luck with your project.
reply
freejazz_RdJ @ 7th Jun 11:35PM:
Re: Blade servers
If you're willing to take a look, Sun has extremely good pricing on their hardware. Their open storage platform, an ZFS more generally, is great to work with and offers great io/sec/$ performance.
Even better if you could get your stack running on Solaris on Sparc. Their T2 based servers offer incredible price/performance for non-HPC loads.
If you insist on a blade server, which makes no sense to me, they also have some offerings.
Cons of the blade server:
Proprietary form factors
Poor storage density unless you buy overpriced storage blades, which sometimes require you to add a SAN switch to the chassis
Extremely expensive startup costs to equip the chassis with PSU, networking blades and such when you've only got a couple blades.
Vendor lock in because of the above proprietary parts.
These and many more are why the blade server never took off like everyone predicted it would several years ago. I'd much rather buy 5-6 rack servers, hook them to a 10G Arista switch and run my storage, internal and external networking over a unified infrastructure than buy any blade server and have to shell out for expensive network blades, PSU's and an external SAN since most blades offer 2 2.5-inch SAS drives at best.
reply
CanerisErik @ 7th Jun 11:47PM:
Re: Blade servers
said by jfmezei :
In terms of remaining on air no matter what, you need to consider what sort of hardware/sofware combination will give you that, especially if you want to do load balancing during normal times so all yoru resources are used to provide the service.
Indeed. If it's any help, we've designed and built, and currently maintain, a system with similar "always on the air" requirements which runs the content for a certain satellite radio channel and phone-in app. While the storage and other requirements are certainly an order of magnitude lower than milnoc's, some of the ideas are the same. Not to sound like an ad for HP again, but we actually don't have blades anywhere, and have found the rack-mounted ProLiants to meet the high availability requirements. You can actually hot swap almost everything: power, fans, drives, NICs, RAM (well, "hot add" is more correct for that one), and you have at least two of almost everything, even on the 1U models.
You would also have more flexibility with storage in a non-blade setup.
--
Erik - Caneris - Internet solutions and more.
reply
Stormlord @ 7th Jun 11:48PM:
Re: Blade servers
For correction I have make a mistake , use 1u server in addition with your san , you don't need more then 2 hdd on vm servers.
reply
anon @ 7th Jun 11:53PM:
Re: Blade servers
I have nothing of value to add to this topic.
However. I'm interested in knowing more about milnoc-TV and what you will be running. If that's ok w/ you.
Also, which cable will be carrying it?
Is it french? English? Bilingual?
Got a page up? Info anywhere?
reply
jfmezei @ 8th Jun 12:08AM:
Re: Blade servers
You may also wish to look into:
»www.apple.com/ca/xserve/
It may be more pricey on the surface, but it does have what is now state of the art in CPU (Nehalem quad core with Quickpath interconnect).
More importantly, it comes with not only Unix (freebsd) but also plenty of OS-X features which may be of value to you as am audio visual content producer. If the extra features are of use/need to you, this may bring the price to competitive level once you add those extras to the barebone systems you'd be getting from HP/Dell/IBM.
The Quickpath (CSI) for Intel and Hypertransport for AMD (both written by ex DEC-Alpha engineers) provide SUBSTANTIAL performance improvements in multi core/multi CPU environments, so don't just look at the Ghz ratings of the CPUs. The Nehalem Intel's also (finally) have a shared cache between cores, and this also greatly imporves performance (you basically have 1 cache instead of each core getting 1/4 of the cache for itself). This means that if core 1 has requested a certain memory location and it is in the cache, when core 2 asks for same location, it is available from cache. In the past, Intel processors were not able to do that so when core 2 asked for memory, it woudl go to the RAM to fetch it even if core 1 had it in its cache.
Quickpath will eventually make it to the IA64 /Itanium. It was originally aimed only at that for high end systems, so it is a bit ironic that we now have 64 bit 8086s with Quicikpath wich outperform what is available on the IA64 side until next year when IA64 gets its next chip. (if Intel doesn't kill off IA64 before then).
reply
milnoc @ 8th Jun 12:08AM:
Re: Blade servers
said by CanerisErik :
Too bad you're not at 151 Front, this "high cost of the dedicated fibre connection" would have been peanuts...many of the large cable companies are here, including Rogers, Videotron, Shaw, and others, so you'd be able to connect to their networks easily.
..!..!..!...
*BING!*
Erik, you've found the solution for me!
I'll set up a pair of automated broadcast clients at 151 Front! They'll download the programming from the Montreal system the same way the smaller cable companies' broadcast clients will download their programming!
As for why I'm insisting so much on blades instead of building the system myself from individual components, I simply don't have the time to figure out how to build and debug these systems anymore. I'm currently working on my own, and will not be able to hire a staff until the channel starts to make some money.
Properly designed blade systems are very simple. Plug everything in, configure them once, and they work. When something breaks down, remove the defective component, and plug in a new component. The IBM Bladecenter alone is best described as a hot-swap-happy-box! :)
In fact, the replacement of nearly every defective component in the chassis (except for the SATA drives which I'll probably purchase and replace myself) will be performed on-site by the manufacturer's support technician. I won't even have to show up at the datacenter.
Currently, the IBM Bladecenter box is less expensive than the HP ProLiant c3000 box by about ten thousand bucks. But this estimate was reached before speaking to a systems specialist who can properly prepare an itemized list of my requirements.
To Curious 1: »www.thecanadianpublic.com . No cable companies have been signed up yet, but I should find at least one this summer, and possibly a whole lot more during an industry meeting scheduled for next September.
reply
jfmezei @ 8th Jun 12:16AM:
Re: Blade servers
Milnoc,
In terms of system management time, you will find that hardware will be the least of your concerns. Software will be.
And you want systems that provide easy remote management. For instance, a telnet/SSH console instead of a VGA console.
One advantage of the Apple solution is that your system comes with the software all pre-installed. So you just configure Apache and you're up and running.
The GUI management tools can be remotely accessed with VNC client on any platform. But you can do most everything with the command line as well.
reply
CanerisErik @ 8th Jun 12:27AM:
Re: Blade servers
said by jfmezei :
In terms of system management time, you will find that hardware will be the least of your concerns. Software will be.
And you want systems that provide easy remote management. For instance, a telnet/SSH console instead of a VGA console.
Agreed with JF again. Systems and software dev will consume far more time, so it's important to have the right resources there.
Speaking of logging into servers and remote management, we don't even do that anymore. We deployed cf last year on our Linux boxes (nearly everything is running Linux) and they just self-manage for the most part now. It's actually quite amazing and worthwhile to learn. Having the guy who wrote a book on it and knows more about it than most, as our senior sysadmin, helps a bit too ;)
reply
milnoc @ 8th Jun 12:35AM:
Re: Blade servers
Unfortunately, the Apple option is out. This system is all about video distribution, and not video editing. All the tools and open source code required to render and distribute video files encoded with just about every codec imaginable are all built around the Windows and Linux platforms.
BTW, all the software required for the broadcast system will be brand new, so there won't be any difference between one hardware platform and another. Anyway, I'm not the one who'll build it. Since my design will be licensed under a Creative Commons license, I'm hoping I can get a lot of software providers participate in the building of this system for free. (yeah, it's a bit crazy)
That reminds me. I may have to revise my blade configuration. I might need two Windows blades and two Linux blades. Windows is great for managing and rendering video files, but Linux is the one that's great as a Web, streaming and BitTorrent server. Luckily, I can easily get four low-end blades for about the same price as two high end blades.
As for remote management, it's all covered. OpenVPN, Remote Desktop, Putty, and TightVNC are all ready to go. I'll be able to control the entire system from my Asus 1000HE netbook.
Enough already! I'm going to bed. Good night! :)
reply
jfmezei @ 8th Jun 01:00AM:
Re: Blade servers
Ok, I am a bit late.
But the the apple option comes with xgrid, which is cluster management tool to manage multiple machines.
»www.apple.com/ca/server/macosx/t···rid.html
Remember that OS-X is Unix (freebsd) with add-ons. So if you have software written for some Unix variance it is easy to port to OS-X, if not already ported.
reply
donoreo @ 8th Jun 06:54AM:
Re: Blade servers
At work we have HP P-Class and C-Class blade servers. All of them are very solid. We have had hard disk failures, but only in the older P's as one might expect. Pop in a new disk and away we go.
The C's are much better for remote management, the Onboard Administrator is much more advanced.
--
The irony of common sense, it is not that common
I cannot deny anything I did not say
reply
pstewart @ 8th Jun 09:00AM:
Re: Blade servers
For what it's worth, we're a Dell Poweredge shop here - very solid servers, very solid. As you have probably seen, there is always varying opinions from folks when it comes to server choices..;)
Someone posted problems with NetApp boxes in an earlier msg - my experience with NetApp has been rock solid for storage (FAS6000, FAS3100 series primarily) ... but NetApp probably isn't what you're after, nor are they "budget friendly"...
--
Nexicom High Speed Internet - »www.nexicom.net/
reply
Guspaz @ 8th Jun 10:50AM:
Re: Blade servers
I'm really not seeing blade servers as a useful solution here. It sounds like you'd be best served with a file server, a gigabit switch, and as many 1U servers as needed. If your performance requirements are low, then the fact that a 1U server can have up to 24 cores (48 logical) at the highest end means that two 1U servers would be able to easily meet your performance requirements and redundancy (unless you want more than single redundancy).
Figure 1U for the switch, 2U for the two servers, 3U for the file server, 1U for the keyboard/mouse/LCD/KVM tray (if it's really needed) and you're at 7U, out of your 10U allotment. Room to grow with another file server or a few more 1U servers.
As for file servers, there are many choices. The simplest to manage is something like the Drobo Pro (gigabit ethernet iSCSI/firewire 800/USB2, 8 drives for up to 16TB current capacity, 3U when rack mounted) which does all the work for you and you just shove drives in it. Costs about $1500 from what I recall. Then there are more robust solutions from storage providers. The highest density is probably from Sun's XFire 4540 (quad opteron, 4U, 48 drives for 96TB capacity). Sun has a bunch of storage solutions in smaller capacity, as do many other enterprise server companies.
reply
milnoc @ 8th Jun 10:57AM:
Re: Blade servers
I'm now awake (mostly). :) We may now resume the discussion.
I'm still checking out Dell, so I don't know yet what's the affordability factor of their line-up.
And Guspaz, an individual components built system is still an option, but I'll have to see if it's still worth it if I can't lease the whole set-up. Leasing equipment has quite a few tax benefits. Also, I don't really want to deal with wires anymore. My prototype server cabinet at the office has already caused a couple of wiring snafus since I built it. Don't want to see the channel go off the air because of a bad Ethernet cable.
I've mentioned in an earlier post that I'm considering a Raid-6 P+Q set-up. Does anyone know which vendor systems currently support such a configuration? At my old job many years ago, one of our clients had actually experienced the failure of TWO hard drives on a Raid-5 array. Having two parity drives in each RAID array would be less stressful for me, especially considering that backing up over ten terabytes of video files is not something I'd want to do on a daily basis.
Yes, the rest of the system data will be backed up daily, maybe even twice daily. I'm lazy. Not stupid. :)
reply
CanerisErik @ 8th Jun 11:00AM:
Re: Blade servers
said by Guspaz :
1U for the keyboard/mouse/LCD/KVM tray (if it's really needed)
With iLO and similar technologies, not really, you just have more network cables and a management VLAN on your switches instead. We don't have KVM cabling, just a crash cart if really really needed.
--
Erik - Caneris - Internet solutions and more.
reply
milnoc @ 8th Jun 11:05AM:
Re: Blade servers
I'll check with the datacenter. They should already have crash carts available. The IBM already has a KVM built in, and I'm expecting it'll be the same for the other manufacturers. Otherwise, how the heck do you set up the system for the very first time?
reply
nitzguy @ 8th Jun 11:34AM:
Re: Blade servers
said by pstewart :
For what it's worth, we're a Dell Poweredge shop here - very solid servers, very solid. As you have probably seen, there is always varying opinions from folks when it comes to server choices..;)
Someone posted problems with NetApp boxes in an earlier msg - my experience with NetApp has been rock solid for storage (FAS6000, FAS3100 series primarily) ... but NetApp probably isn't what you're after, nor are they "budget friendly"...
No, they aren't definitely budget friendly :)...I was just commenting because of my job how it wakes me up at inopportune times..It is quite a beast though...I'd take pictures but my company frowns upon taking pictures
of their equipment for security reasons :).
And, I'm just the guy at the end of the line who looks at the pretty lights and whatnot...remote admin is done in a far off land, I'm just the monkey who checks on it to go "yup, it says FAULT"...
But I'm sure for the most part since it does about 3000 transactions/sec that its very reliable and does the job well.
reply
milnoc @ 8th Jun 12:17PM:
Re: Blade servers
Anyone know how I can get a preliminary cost assessment on a Dell blade system without having to call a Dell rep? Both IBM and HP provide me with either a price list or some software tools to help me get an idea on what their systems will cost. But Dell's site says to "Call us".
Call who? India? The Philippines? Mike from Canmore? :)
Unless the salesperson has a referral from someone I know, I'm not sure I'd want to trust them with the configuration of a $20K-$30K system. With IBM, I'm already speaking with my former employer's supplier, an individual I trust to get me exactly what I need, at least where it concerns IBM equipment.
reply
Guspaz @ 8th Jun 12:50PM:
Re: Blade servers
said by milnoc :
And Guspaz, an individual components built system is still an option, but I'll have to see if it's still worth it if I can't lease the whole set-up. Leasing equipment has quite a few tax benefits. Also, I don't really want to deal with wires anymore. My prototype server cabinet at the office has already caused a couple of wiring snafus since I built it. Don't want to see the channel go off the air because of a bad Ethernet cable.
There isn't a huge wiring difference between a blade server and rackmount servers. You only have two cables coming out of each anyhow, power and ethernet. Ethernet plugs into your switch (which in your blade server may be integrated, although you'll still need to plug the file server into it) and power plugs into whatever PDU you're using (I don't think vertical strips necessarily need to consume a rack unit).
Most rackmount servers include at least two NICs, so you can always use both if you want. But Ethernet cables tend not to suddenly go bad in a static installation where nothing ever moves.
If you have two servers, a file server, and a switch, you'd simply plug your four ethernet cables into the switch and that's it. You might want to set up some vlans, but that's more of a software config thing.
I've mentioned in an earlier post that I'm considering a Raid-6 P+Q set-up. Does anyone know which vendor systems currently support such a configuration? At my old job many years ago, one of our clients had actually experienced the failure of TWO hard drives on a Raid-5 array. Having two parity drives in each RAID array would be less stressful for me, especially considering that backing up over ten terabytes of video files is not something I'd want to do on a daily basis.
Both solutions that I mentioned (Drobo Pro and Sun's stuff) supports double-parity. Drobo isn't RAID-5 or RAID-6, but their own stuff, but double-parity can be enabled. Typically on Sun's hardware, you'd use Solaris/ZFS/RAIDZ, and they have RAIDZ2 which provides double parity (RAIDZ/RAIDZ2 is quite a bit more reliable than software RAID-5/RAID-6).
Most vendor solutions should support RAID-6, it's pretty standard. That said, I've got no direct experience here. It's just a subject I'm interested in and have done a fair bit of reading on.
Yes, the rest of the system data will be backed up daily, maybe even twice daily. I'm lazy. Not stupid. :)
You should probably be backing up that 10TB array of yours too. RAID is not a backup solution, it simply covers hardware failures. It doesn't (with the exception of ZFS/RAIDZ which do per-block checksums to ensure integrity) care about data corruption since any changes are simply mirrored. It doesn't care about data being erased due to hackers. It doesn't care about data lost due to accident. It's for hardware failure and *nothing* else. It's not particularly difficult or bandwidth-intensive to do offsite backups. Simply set up a matching storage array at your office and do periodic rsyncs to keep a local copy up to date. If you use ZFS or any other filesystem that supports snapshots, you can even ensure that no data changes while doing your rsync.
reply
milnoc @ 8th Jun 01:40PM:
Re: Blade servers
The +10TB repository is basically a storage library for the channel's audio/video files, including streaming/BitTorrent files. Whenever a file is stored in the repository, a copy will be sent to the office server as well. The files in the repository never change; they're either added or deleted. So a regular backup schedule for the big drive is unnecessary since there will always be a way to quickly reload at least the following week's programming schedule (the programming will be sent to the broadcast clients up to a week in advance in case the repository goes down for any length of time).
As for the blade chassis, it'll only need about six cables. Four for the power supplies, and two for the Internet. Everything inside the chassis (including mass storage) is interconnected through two sets of pathways in case one pathway goes down.
The RAID-6 P+Q must be a hardware solution, and not a software solution. I'll still check out Drobo's offerings however. I'm aware they've been making some decent products for a while, and eventually I'll run out of room on the blade and will have to move the streaming/BitTorrent files elsewhere (those are the only files that will always be preserved).
Another reason for using a blade system is for marketing purposes. I might get a decent sponsorship deal out of this, especially if I do like Rocky and record the set-up and installation at the datacenter. Unlike other companies that prefer to keep everything a secret, I want to show off my "digital bling!" :)
reply
Guspaz @ 8th Jun 02:02PM:
Re: Blade servers
said by milnoc :
As for the blade chassis, it'll only need about six cables. Four for the power supplies, and two for the Internet. Everything inside the chassis (including mass storage) is interconnected through two sets of pathways in case one pathway goes down.
You're paying a pretty hefty premium to get everything in one chassis.
The RAID-6 P+Q must be a hardware solution, and not a software solution. I'll still check out Drobo's offerings however. I'm aware they've been making some decent products for a while, and eventually I'll run out of room on the blade and will have to move the streaming/BitTorrent files elsewhere (those are the only files that will always be preserved).
Why does it need to be hardware? Hardware solutions aren't any more reliable. In fact, in some ways they're *less* reliable. If your RAID card dies, it could be a nightmare (or impossible) to migrate the data to a different card/server. RAIDZ, for example, doesn't have this issue, and offers per-block checksumming via ZFS. I believe ext4 is supposed to offer that too, although not integrated with RAID and I wouldn't trust ext4 with enterprise stuff for a while.
In short, some software RAID solutions are likely more reliable than many hardware RAID solutions. Also, note that Drobo is a software RAID solution.
Another reason for using a blade system is for marketing purposes. I might get a decent sponsorship deal out of this, especially if I do like Rocky and record the set-up and installation at the datacenter. Unlike other companies that prefer to keep everything a secret, I want to show off my "digital bling!" :)
Bling is nice, but spending thousands of dollars on a blade chassis (and that's before you add any actual blade servers) is a waste of money if your blade chassis alone would cost almost as much as all the other hardware in a normal setup.
reply
milnoc @ 8th Jun 03:10PM:
Re: Blade servers
A blade chassis can hold multiple RAID controllers. If one breaks down, the other takes over without interruption. Same goes for the network switch (two of them), power supplies (FOUR of them), case fans (four) and even the blades (two mirrored drives in each blade). All of it hot-swappable.
And yes, there is a hefty premium because I'm pretty much buying two of everything. But when you're trying to sell a television channel to the cable companies, they INSIST that your system have redundant safeguards built in. If my channel goes down for any reason whatsoever even for five minutes, THEY'RE the ones who will get the ton of phone calls from their customers complaining that the channel is down! That's assuming my Canadian channel becomes popular at all of course. :)
A blade system covers all of my bases both on the technical side and on the marketing side. And as long as I lease the thing instead of buying it, the monthly cost will be well within my company's operating budget. Purchasing the system outright, even one I build myself, might compromise the company's financial situation, which for a start-up company can only be pushed so far.
The bling is necessary. There's no way around it, both on the technical side and on the marketing side.
reply
Guspaz @ 8th Jun 05:04PM:
Re: Blade servers
said by milnoc :
A blade chassis can hold multiple RAID controllers. If one breaks down, the other takes over without interruption. Same goes for the network switch (two of them), power supplies (FOUR of them), case fans (four) and even the blades (two mirrored drives in each blade). All of it hot-swappable.
While I can't speak as to the specific IBM setup, I've never heard of hot failover between RAID controllers. In fact, as far as I can tell, IBM's documentation only mentions the switches, power supplies, blowers, and midplanes as being redundant. There's a passing reference made to installing a second RAID controller for redundancy, but it's not clear that it can actually do failover (as opposed to manual intervention to switch over), and there aren't any details as to how you might migrate the RAID array config between controllers. It also seems that you can only do hardware RAID with SAS drives. That's not necessarily a problem, but SAS drives currently seem to max out at 1TB (as opposed to 2TB for SATA), and cost a bit more (a quick check showed 1.5-2.5x the price of SATA).
Also, their RAID controller doesn't seem to support RAID-6, only RAID-5.
However, you can probably access any drive from any blade, so you can have one blade run Solaris or BSD and use ZFS/RAID-Z2 to get the double-parity you're looking for along with the other benefits. Although be warned that you can't add drives to a RAID-Z array, so you'd need to fully populate all 12 bays at the start. You can then swap out larger drives in the future. ZFS pools support adding drives, but then there's no redundancy.
And yes, there is a hefty premium because I'm pretty much buying two of everything. But when you're trying to sell a television channel to the cable companies, they INSIST that your system have redundant safeguards built in. If my channel goes down for any reason whatsoever even for five minutes, THEY'RE the ones who will get the ton of phone calls from their customers complaining that the channel is down! That's assuming my Canadian channel becomes popular at all of course. :)
A blade system covers all of my bases both on the technical side and on the marketing side. And as long as I lease the thing instead of buying it, the monthly cost will be well within my company's operating budget. Purchasing the system outright, even one I build myself, might compromise the company's financial situation, which for a start-up company can only be pushed so far.
You've gone quite a ways from wanting to put an Eee PC in a datacenter to handle everything ;)
reply
milnoc @ 8th Jun 06:02PM:
Re: Blade servers
said by Guspaz :
You've gone quite a ways from wanting to put an Eee PC in a datacenter to handle everything ;)
That's just the local broadcast client for the smaller cable companies. And it's still an option if there are any takers! Staples is selling the 900HA for only $309! :D
I'll double-check the IBM Raid controller specs, but I do believe the controllers will switch over if one fails. In fact, I believe they require you to order two of them. They may still be on special for $2341. EACH! :)
reply
jfmezei @ 8th Jun 06:38PM:
Re: Blade servers
Mr Milnoc,
A blade solution will work, just as would stringing up enough old PCs.
If you have multiple OS instances access your storage, you want the RAID to be inside the storage array (centralised) and each OS instance then connects to the storage array with fibre channel or some other technology. If each instance has its own RAID card, which instance will correct a parity error if it happens ? Who controls who can write to a sector on the disk at the same time ?
Don't *rely* on the vendors to tell you what the best solution is. They will quote what you ask. They have long ago stopped having consultants study your needs and propose the best solution (unless you are going to spend half a million or more).
Another thing to consider: if you have multiple instances, will you want a shared system disk or have multiple system disks. This matters when the time comes to do software upgrades.
reply
Guspaz @ 8th Jun 06:55PM:
Re: Blade servers
The IBM BladeCenter uses dedicated RAID cards such that any blade would go through them. I'm still saying that a software RAID solution running on one of the blades (a lower performance blade designated as a service module, perhaps, to handle file serving and routing and such) would be a more reliable solution, albeit one with less redundancy. Although ZFS pools can be recovered by another box if the host system dies.
reply
milnoc @ 8th Jun 07:09PM:
Re: Blade servers
The RAID-5 array will become shared storage, shared by all the blades. Each blade will also have their own pair of hard drives, configured as RAID-1. The blades will boot off their internal RAID-1 array, then connect to the RAID-5 array to access the video files.
I've checked the BladeCenter's RAID controllers. You DO have to buy two of them. You can't just buy one. So if one breaks down, the other one kicks in. They even come with battery modules so that anything in the cache will be preserved for 72 hours following a power failure.
Now my only problem is the hard drives. 1 TB SAS drives, $974 each. Times twelve, $11,688. Unfortunately, the SATA option isn't available.
The current price for the IBM Bladecenter S with everything included adds up to a grand total of...
Thirty-three thousand dollars.
Yikes.
I have some mighty big nuts to squeeze to successfully bring down that price. Luckily, we're in a recession. I might accomplish something here.
Time to research HP's offerings and pricing.
reply
Guspaz @ 8th Jun 07:27PM:
Re: Blade servers
A quick check showed 1TB Seagate SAS drives selling for $255. I believe that IBM will sell you the drive chassis bare. Also, a copy-on-write file system with double parity and per-block checksums is more reliable than hardware RAID-5 even with battery backups, doubly so if a 1U UPS is used to clear out the buffer (CoW maintains a consistent state by making all writes atomic). For one thing, RAID-5 can't handle two drives failing. For another, it doesn't completely handle data corruption.
reply
milnoc @ 8th Jun 08:01PM:
Re: Blade servers
The price wouldn't be so bad if total storage approached 20 TB. But with only 10 TB of storage, that's a high price to pay.
IBM has always been problematic with their hard drive pricing. I can understand the reasoning behind the pricing of the other components; they don't sell that many individual units when compared with the retail market of many components. But hard drives? They're all made at the same handful of Chinese factories! Their mark-up is way out of line.
IBM hard drives. The Monster Cable of the computer industry. :)
And yes, you can buy the system without the hard drives. But I'll still want to squeeze their nuts on the price. Now where's my nutcracker? :D
An idea. I'll check out Asus. I have two of their RS100-X5/PI2 servers in my server rack, each built for under a grand. I might not actually buy into it (the entire system may still end up costing ten grand and may not have all the needed safeguards), but it'll get me some pricing info I can use as leverage.
Aren't we having fun shopping for computer stuff? :)
reply
freejazz_RdJ @ 8th Jun 08:05PM:
Re: Blade servers
I'll second Guspaz on being very pro-ZFS/RaidZ. The Sun storage solution, purchased as their open storage product or cobbled together (I run a NAS with 6X1TB, RaidZ2) offers a whole host of advantages over the standard raid-5/6 cards.
At the simplest level, you dispose of proprietary hardware and it's costs, you have unlimited scalability (both in terms of maximum volume size and ports... as many ports as you can cram into the host) and portability to other systems. I can take my 6 disks and move them to a new Solaris machine and bring them up easily.
ZFS is brilliant and one of the few reasons I use Solaris, the others being it's great TCP/IP stack, containers/zones, I like Sparc T1/T2 and lots of OSS/BSS software demands solaris on NEBS servers.
reply
milnoc @ 9th Jun 09:41AM:
Re: Blade servers
I'll definitely check Sun out. The storage solutions offered by the blade providers all appear to fall short on some of the items that I need, most important the ability to use cheaper and higher capacity SATA drives. The performance hit encountered with SATA drives won't be a problem since the mass storage unit will be used exclusively to distribute video files over P2P, meaning it won't be accessed as frequently as one would normally expect.
I might still use a Blade enclosure though. I just won't use any of the storage "solutions", which seem to cause more problems than solve them. The individual blades aren't that much more expensive when compared with standalone servers, and they're extremely easy to manage.
To search and research. :)
reply
anon @ 10th Jun 11:08AM:
msg deleted
deleted by a moderator
reply
jfmezei @ 10th Jun 11:25AM:
Re: Blade servers
>most important the ability to use cheaper and higher capacity SATA drives
I know you refuse to look at Apple but...
»www.apple.com/server/storage/
Their VTRAKRaid solution uses SATA drives. (but you are restricted to drives qualified by Apple to ensure they work properly with the software, and you'll find this is the case for any serious SAN vendor)
And I know you feel you need windows for broadcast but...
»www.apple.com/xsan/deployments/b···ast.html
reply
warren @ 10th Jun 11:54AM:
Re: Blade servers
You also have to watch out for the Higher capacity SATA drives. The rebuild time for a failed array is exponentially bigger for a 2TB drive than it is for a 500GB drive. Also there's a higher probability of failure during a rebuild of another SATA drive, as it's peaking it's transfer rates.
There was an article that had some very good information with regards to the above problems, but I'm unable to find it again (we went through this ~ 6 months ago for a storage solution at my previous employer).
Just a heads up.
reply
El Quintron @ 10th Jun 12:15PM:
Re: Blade servers
Linux Mag keeps running ads for AoE products (sATA over Ethernet) you keep adding drive as more storage is needed and it's a relatively inexpensive solution out of the box.
--
Working to bring you closer to a Bell and Rogers free household.
reply
milnoc @ 10th Jun 12:19PM:
Re: Blade servers
said by warren :
You also have to watch out for the Higher capacity SATA drives. The rebuild time for a failed array is exponentially bigger for a 2TB drive than it is for a 500GB drive. Also there's a higher probability of failure during a rebuild of another SATA drive, as it's peaking it's transfer rates.
That's why I was hoping to use RAID-6 P+Q. It uses two parity drives instead of one, meaning you'd have to lose three drives before the entire volume is lost. It's like having a preloaded backup drive on stand-by all the time.
And as I've mentioned before, I know one former client who suffered the loss of two drives in a RAID-5 array. The drive contained dynamic data, so of course it was already backed up daily. It was still a bitch re-entering the day's lost data, plus all the data that wasn't entered while the server was down.
I'll check out Apple's offerings, but they'll have to be extremely flexible dealing with various exotic codecs. Also, one of my publicity stunts will be to build an automated broadcast appliance out of a netbook.
Imagine! A nationwide television broadcast originating from a tiny $300 computer! How freaky is that? :D
reply
DSL_Ricer @ 10th Jun 12:42PM:
Re: Blade servers
On thing you may also want to check out, instead of getting redundant raid controllers, is just getting redundant arrays:
Buying two complete dual Xeon systems with 12GB ram and 15x1.5TB disks + 2x24 port managed switches can be had for under 15K$ shipped.
Then you can stick something like drbd on it (»www.drbd.org/) and, voila, instant redundancy.
Since you have duplicate everything, that should mean no single point of failure.
reply
milnoc @ 10th Jun 01:04PM:
Re: Blade servers
The dual array solution is already under consideration. It's an essential item if the chosen single RAID array doesn't have enough redundancy built-in.
reply
Guspaz @ 10th Jun 04:11PM:
Re: Blade servers
Sun's storage solutions use ZFS+RAIDZ (both of which are Sun technologies), so keep in mind that you don't need Sun's hardware to use ZFS+RAIDZ. You can do it on any hardware, even the IBM bladecenter. All it requires is an OS that supports ZFS, such as Solaris or BSD (both Unix rather than Linux).
So the IBM BladeCenter with a server blade dedicated to RAID rather than the RAID controllers would be the equivalent.
Sun's biggest solution is the Sun Fire X4540, which has 48 SATA drive bays for up to 96TB of storage in a 4U space. But that's probably outside your budget since it starts at $22k USD for 12TB of storage (48x250GB). Subtract 2 drives worth if you're using 2-drive parity.
It doesn't necessarily make sense to put all 48 drives in a single RAID array either. Because if you put in 48 drives and you want to upgrade the array to a larger drive size, you'd have to replace all 48 drives before you could properly grow the array, I believe. If you broke up the 48 drives into, say, 4 smaller arrays (10 data + 2 parity each), then you'd only have to replace 12 drives to upgrade the storage space of an array.
But as I said, 48 drives is probably incredible overkill for you.
reply
milnoc @ 10th Jun 05:07PM:
Re: Blade servers
48 drives. That's gonna be a lot of fun to assemble! :)
And yes, I'm aware I don't need Sun's server boxes to get the job done. However, I've already done a price check on a Sun storage bay combined with an IBM BladeCenter "E" chassis (blades and networking only), and it's not practical.
I've done all my preliminary calculations, and I've come up with the following results.
- A custom designed system (no blades) with one 15 TB storage array will cost about $10K.
- A custom designed system (no blades) with TWO 15 TB storage arrays will cost about $15K.
- A top-of-the-line pure IBM Bladecenter "S" set-up with a 10 TB storage array (SAS hard drives only) retails for $33K (are they nuts?).
I'm currently pushing my sales rep. very hard to lower the price of the IBM box down to just over $20K. This box will feature my 15 TB SATA array, two quad-core blades, and redundant components throughout.
If IBM says "no", I'll choose instead the first custom designed system. There will be a lot of work involved getting everything up and running properly, probably more work than I want to deal with, but I won't have much of a choice. The pricing of the other manufacturers' blade systems is comparable to IBM's pricing, so there's no point in researching them any further especially since they make it impossible to configure and price a system on-line.
reply
jfmezei @ 16th Jun 01:54AM:
Re: Blade servers
Sorry to bring this up again.
I hate to support HP, but to be fair:
They came out with a new line of servers, the SL 6000 which are basically 2U cabinets in which you can fit 2 * 1 U servers. Shared power supplies and fans provide redundancy, and this is apparently lower cost than a real blade.
»h18004.www1.hp.com/products/serv···ent.html
reply
milnoc @ 16th Jun 02:20PM:
Re: Blade servers
Interesting, but what's the price difference when compared with two ordinary 1U servers?
Yes, it's a silly question to ask considering I'm looking at paying $20K for an IBM blade. But it's spread over three years though! :)
reply
anon @ 17th Jun 08:46AM:
Re: Blade servers
Hey Milnoc...this wouldn't happen to be your project, would it?
»www.digitalhome.ca/content/view/3765/279/
reply
milnoc @ 10th Sep 10:03AM:
Re: Blade servers
Whoops! Didn't realize someone asked me a question a while back. :)
Anonone, that's definitely not my channel. The terms of my CRTC license are so broad, the only difference between my channel and a national broadcaster are the OTA transmitters. The link you provided is for a different channel with much higher content genre restrictions.
Also, my personal Canadian content requirements are much, much higher. 70% during the first year, 90% within three years. However, the CRTC imposed on my channel the traditional 15%-25%-35% yearly cancon requirements. So I have plenty of maneuvering room.
An update on the BladeCenter. It look like the deal will work out.
I've already checked with my regular "el-cheapo" supplier Tiger Direct, and the price for an equivalent configuration using separate components that I have to assemble myself was about $10K.
I passed this information to my IBM supplier. He came back with a much more agreeable quote than the previous BladeCenter quote.
Gotta love competition!
reply
Guspaz @ 10th Sep 01:58PM:
Re: Blade servers
Yikes! If TigerDirect is your idea of el-cheapo, I don't want to hear your idea of high-end! While I haven't checked their prices recently, last I did their prices were a ripoff, their shipping was expensive, and they charged sales tax despite not being in Quebec. Which is because they're not Canadian, but American shipping over the border.
reply
milnoc @ 10th Sep 02:37PM:
Re: Blade servers
My idea of high-end is IBM. :D
They WERE pretty bad just a few short years ago. But with the proliferation of online shops that are located in or ship to Canada, they had to adjust their prices pretty dramatically. You also have to be picky, and comparison shop as much as possible before buying anything from them. But the deals are there!
I also have a corporate account with them that gives me further rebates on my purchases. ;)
reply
anon @ 10th Sep 06:00PM:
Re: Blade servers
Acually tigerdirect have a shop in toronto, but canada computer still give me better bangs for bucks for brick and morter
reply
twizlar @ 10th Sep 07:42PM:
Re: Blade servers
If I need it in a hurry, Canada Computers, FrontierPC, or Tiger Direct work fine. Otherwise most things come from newegg.ca now. Not very often can their pricing be beat for most of the hardware I buy.
--
Broadline Networks Inc.
reply
jfmezei @ 10th Sep 09:56PM:
Re: Blade servers
Note that shops like Tiger Direct and even Office Depot have their published web prices and have internal prices. Can't remember whom I bought a cisco router from, but I was miffed that Cisco doesn't allow canadians to buy from US outfits. So I looked around the canadian outfits that carried the model I needed and the price was steep but it had a "call in" image on the product. So I called in, the guy confirmed the price, and I noted it was really not competitive with the US prices, and he sais "let me see what I can do", and magically, the price came down substantially.
reply
NeTwOrKDawg @ 11th Sep 12:27AM:
Re: Blade servers
said by jfmezei :
Note that shops like Tiger Direct and even Office Depot have their published web prices and have internal prices. Can't remember whom I bought a cisco router from, but I was miffed that Cisco doesn't allow canadians to buy from US outfits. So I looked around the canadian outfits that carried the model I needed and the price was steep but it had a "call in" image on the product. So I called in, the guy confirmed the price, and I noted it was really not competitive with the US prices, and he sais "let me see what I can do", and magically, the price came down substantially.
Yes, it is called MSRP. MSRP is always higher than wholesale cost. By how much depends on the product. I know from experience that MSRP is usually about 30-40% higher than wholesale cost with Cisco.
reply
milnoc @ 11th Sep 09:20AM:
Re: Blade servers
said by NeTwOrKDawg :
I know from experience that MSRP is usually about 30-40% higher than wholesale cost with Cisco.
For products from Monster and Bose, add a few zeroes. :)
reply
milnoc @ 12th Sep 05:56PM:
Re: Blade servers
Crap.
I just found a serious problem with the order. The system is missing the RAID-5 shared storage support altogether. On top of that, the order might be missing the server modules required for the blades to communicate with the storage.
Because of scheduling issues and other commitments, it's going to take me at least two weeks to sort out this mess, which means I'll definitely miss my projected launch date of October 13. It'll have to be moved to November 1st at the very least, and even that may not be enough time to try and sort out this mess or order alternate equipment through a different supply chain.
Mistakes do happen of course. But in this case, it happened with people that have been reliable for nearly twenty years. And considering the amount of money that's involved here, it's pretty serious.
I'll let you all know what happens.
reply
twizlar @ 12th Sep 05:57PM:
Re: Blade servers
You should be able to pick up whatever you need in 2-3 days from any larger reseller/wholesaler.
--
Broadline Networks Inc.
reply
milnoc @ 12th Sep 06:00PM:
Re: Blade servers
Except that the mistake may cost me an extra $7,000 in the configuration. That blows the entire budget for the system.
TigerDirect is starting to look good.
reply
twizlar @ 12th Sep 06:08PM:
Re: Blade servers
lol. Avoid at all cost :). Tell me what you want and i'll check some prices for you.
--
Broadline Networks Inc.
reply
lester @ 12th Sep 06:09PM:
Re: Blade servers
My friend keeps telling me shopbot.ca is a good site to compare prices haven't tried it myself but its probably worth checking out
reply
twizlar @ 12th Sep 06:12PM:
Re: Blade servers
They don't really have things like blade centers on shopbot.
I take it back, they have some ibm blade stuff.
reply
lester @ 12th Sep 06:17PM:
Re: Blade servers
.
reply
milnoc @ 12th Sep 06:18PM:
Re: Blade servers
Let me first work things out with my supplier. He did get most of the configuration set up properly. It's only with the storage management (not the storage itself) that it all fell apart.
As for what I need specifically, it's two IBM BladeCenter S SAS RAID Controller Modules, part number 43W3584. They cost on average $3,500 each, and the BladeCenter won't accept just one of them.
I might also need the matching SAS modules for the blades, but I'm not sure. I want the RAID to work like a NAS, which was specified during the order. The modules might not be required for such a configuration.
reply
twizlar @ 12th Sep 06:19PM:
Re: Blade servers
lol, looks like they index most of the retail sites.
reply
twizlar @ 12th Sep 06:29PM:
Re: Blade servers
Tell your ibm guy to give you a better price on it. They are listed retail for around $3300, a couple places have them for under $3000.
--
Broadline Networks Inc.
reply
milnoc @ 12th Sep 07:24PM:
Re: Blade servers
Careful. They might be refurbished units.
I might need to do some revising of the system first. IBM's Redbook on the S chassis indicates that shared volumes are possible, but it doesn't properly explain how to do it except by indicating you can use VMWare to do it. There's no indication whatsoever that the RAID controller can behave like a NAS.
I've had an idea. If the RAID array can't properly act as a Linux EXT3 formatted shared storage device except by going through a separate blade, then I might as well keep what I have (including the much cheaper SAS Connectivity Module part number 39Y9195 at about a grand), and install whatever I need to manage the individual discs in the system as a RAID array.
Luckily, I already have two blades in the system. So I can use one as a RAID manager and see how it goes from there. And since I already have three 1 TB drives in the array, I can rig a hardware based RAID 1E configuration via the IBM connectivity card in the blade, which would be a reasonable compromise over RAID-5, and still give me the needed hot swap capabilities.
This non-RAID-5 set-up also has the advantage of supporting commercially available SATA drives, which are much, much cheaper than SAS drives. They'll be perfect for the archive repository.
All I need to complete the setup (if it isn't already included in the order) is the blade's connectivity card, either part 39Y9190 or part 43W3974.
During the first months of operation, the entire system will be in a test mode during which no advertising will be sold. So if the RAID blade crashes, it's not serious on the revenue stream, but it will still shut down the television broadcast.
Anyway, this new set-up is no worse in complexity and "unreliability" than a "separates" configuration. And surprisingly, the price difference isn't that huge. The biggest savings however is with the datacenter costs. A separates system would have pushed the limits and probably surpassed the height of the 1/4 rack enclosure, increasing the monthly costs by at least $200 per month. The IBM S chassis however, at only 7U in height, leaves me plenty of room to spare.
I'll talk to my rep. after he comes back from his vacation in a week. My configuration may be all right after all, but it's definitely not what I was expecting.
reply
NOCMan @ 13th Sep 12:46PM:
Re: Blade servers
Not sure if you've settled on anything, but I would suggest hardware redundancy. Two servers in the DC, but on different switches in the DC, on separate UPS supplies voltage rails whatever. This way if your server fails, they do maintenance on the power, UPS failure, it all fails over to the secondary server.
Where I work we are very anal about redundancy and we go through a lot of crap to make sure things work correctly, but in the end we suffer very few if any outages of our services to customers. All 80+ million of them.
FWIW, we use IBM, HP, Dell, and Sun servers they have all their quirks, but your system is only as good as the redundancy. They'll all screw up at one point and if you have nothing to fail back on then yes you're screwed.
--
Play a Death Knight?
www.theebonhold.com
reply
milnoc @ 13th Sep 02:10PM:
Re: Blade servers
The redundancy is already in the works. This initial system will be the proof of concept for the automated broadcast system. Once that's working reasonably well, I'll add in the extra hardware required to provide all the necessary redundancy, at least within the same chassis.
I might simply use the blades themselves as the RAID controllers. At just over a grand each, it's cheaper to buy two of those along with an extra SAS connectivity module than it would cost to purchase only one of those hyper-expensive dedicated IBM RAID controllers which might not be shareable the way I want.
BTW, IBM's year-end is coming up. If the year was bad, they'll be putting up a ton of promotions just to sell as much stuff as they possibly can. I might be able to get a few deals and properly complete the hardware configuration which will most likely consist of:
- One IBM BladeCenter S chassis
- Four HS12 blade servers (currently have two)
- Two SAS Connectivity modules (currently have one)
- Six 1 TB SAS hard drives (currently have three, might get more than six)
- Two drive bays (currently have one)
- Two Nortel 1 Gbps network modules (currently have one)
- Four power supplies (already have four)
If I can get the rest of the equipment for under five grand, I'll have TWICE the power required to multicast multiple broadcast feeds, run the BitTorrent servers, support the database, and so on. I'll eventually need all of that power considering I'll be running a second television channel off the same system.
BTW, this system will NOT be used to host the Web site. At only seven bucks per month, I can let GoDaddy take care of that for the time being. It's not the fastest solution out there, but it's definitely the cheapest and the easiest to maintain for now.
The ultimate goal? Sell the broadcast system as a service to television stations that don't want to purchase and support their own hardware infrastructures.
reply
milnoc @ 13th Sep 02:59PM:
Re: Blade servers
BTW, thanks to everyone for the ongoing suggestions on alternate configuration options both with and without the IBM equipment. They really helped me find cost effective solutions for my system configuration.
This is actually going to work!
reply
milnoc @ 25th Sep 10:50PM:
Re: Blade servers
It's not looking good. IBM may be out of the picture if they don't get their act together by noon next Monday.
The problem? I can't get an answer to a simple question. And it's required to complete the order. Will a single-port SATA drive (the kind you buy for your own computer) fit properly in the storage module, and function properly with the SAS Connectivity Module?
Honestly, I get the feeling I know more about the capabilities of IBM's equipment than IBM! I've read through the entire Redbook for the BladeCenter S chassis! I can almost recite the individual part numbers! I've been busy correcting many mistakes in the order for the last two weeks!
As I've said, if they don't get their act together by noon next Monday, and deliver the merchandise by October 13th, there's no point in pursuing this. The delays are already costing me money, and it's doubtful IBM will ever compensate me for the losses.
I do have a plan B however. Two Asus RS120-E5/PA4 barebones servers, fully loaded with Quad-core processors, 4 GB of memory, and up to 3 TB of RAID storage each. I'll have to buy a lot of support equipment (cables, switches, tie-wraps) and do a lot of work assembling it all, but at least I can get most of the equipment delivered before the end of next week, and installed before my October 13 deadline.
My biggest disappointment is with my long time supplier. He's always been reliable. But this time, the order process has become a disaster.
Here's hoping things get better over the weekend. I'm really feeling angry and frustrated at this time. I don't need this bullshit.
reply
jfmezei @ 25th Sep 11:25PM:
Re: Blade servers
If you were a bank, such a request would have resulted in the IBM rep assigned to the bank getting a demo server delivered to your office with himself or a tech making sure all your questions were answered.
I've seen a such a rep stay at his bank office (yeah, he had an office in the bank's data centre) well past office hours on a friday to ensure that he could solve any problems the staff could have to get a pilot project going before end of weekend, and he even went back to the IBM offices to get more memory for that machine at 20:00 when the bank staff had problems. Those were just a couple of PCs running OS2. But it was to be a high profile project if it went through.
Unfortunatly, that IBM rep now has a VP job at: Bell Canada.
Large companies such as IBM are not really equipped to deal with small customers.
reply
milnoc @ 26th Sep 02:01PM:
Re: Blade servers
What's ironic about your last statement is that the BladeCenter S chassis was designed specifically for small businesses. But it seems to be impossible for IBM to sell it within a reasonable amount of time. It's been four weeks already!
I'll say it again. I don't need this bullshit.
reply
freejazz_RdJ @ 26th Sep 02:15PM:
Re: Blade servers
My beef with many of the chassis systems (I've seen Rackable, Sun, HP and IBM blades) is that it is very difficult to figure out exactly how the IO breaks out from the blade. Is the module a pass-through? Is it non-blocking? Do I need module X for it to boot even though I won't actually use it? Is the midplane passive or a switched fabric?
Frankly, the only blade systems I've ever had a positive vibe with have been those that ship fully integrated into a cluster, like a sun 6000 or 6048 system for Oracle real application clusters. Plus the port costs on the network blades are insane... far higher than you'd pay for equivalent PCI-E add ins, or onboard 10Gige.
reply
milnoc @ 27th Sep 05:39PM:
Re: Blade servers
Ok. Need help here. I need a rackmount Gigabit Layer-3 switch with NAT support. A router if you prefer. My Internet connection will have a static IP address, but I wouldn't mind support for a /30 subnet for later use. The rest of the internal network will function off a private subnet (192.168.x.y).
Any suggestions? The product documentation I'm finding on-line is highly cryptic.
reply
pstewart @ 27th Sep 05:55PM:
Re: Blade servers
For what it's worth, we are 100% Dell Poweredge here with varying hardware ranging from 800 series up to R905 with disk arrays etc. They have all been rock solid over the 10+ years of running them here.
--
Nexicom High Speed Internet - »www.nexicom.net/
reply
milnoc @ 27th Sep 06:09PM:
Re: Blade servers
Sorry. I'm out of time. I can no longer waste a whole month dealing with another large corporation. I have to get my gear from providers that can deliver most of the required components by the end of the week. Every day I don't have the gear installed after October 1st will cost me money, the biggest drain occurring after October 13th.
reply
jfmezei @ 27th Sep 06:24PM:
Re: Blade servers
You may have had more chances dealing with a reseller instead of dealing with IBM directly. They can spend a bit more time with customers of your size.
In terms of your router, when you get to the gigabit throughput size, you'te dealing with enterprise products. A consumer product may have gigabit switches, but not gigabit throughput through the router.
reply
milnoc @ 27th Sep 06:27PM:
Re: Blade servers
I did deal with a reseller.
As for using enterprise products, that's not a problem. What I need are make and model numbers. I'll be ordering it tomorrow afternoon if I get no response from my reseller by noon.
reply
twizlar @ 27th Sep 06:30PM:
Re: Blade servers
How much bandwidth do you want to be able to push with the switch. You are looking at pretty big numbers for a layer 3 routed wirespeed gigabit switch if thats what you want. Are you sure you want to actually run NAT? Are you talking true 1:1 NAT? or PAT?
--
Broadline Networks Inc.
reply
milnoc @ 27th Sep 06:36PM:
Re: Blade servers
I want the same as what I can get with a consumer router: NAT, private network, port forwarding. If no one can name me any products that could fit the bill, I might as well get a cheap server, load it with a network management Linux distro, and be done with it. But I'd rather not go that route because it'll take me a couple of days at the very least to set it up and configure it properly. More wasted time.
I'm going to be installing this at a datacenter. I ain't stuffing consumer products in that rack.
reply
milnoc @ 27th Sep 06:38PM:
Re: Blade servers
As for bandwidth, I'll only have a 10 Mbps connection for starters. But that can quickly increase to 100 Mbps once I find a few cable distributors for my channel.
The system will not be used for regular Internet stuff such as Web and mail. It'll be used for BitTorrent distribution and live feeds.
reply
pstewart @ 27th Sep 07:04PM:
Re: Blade servers
said by milnoc :
I want the same as what I can get with a consumer router: NAT, private network, port forwarding. If no one can name me any products that could fit the bill, I might as well get a cheap server, load it with a network management Linux distro, and be done with it. But I'd rather not go that route because it'll take me a couple of days at the very least to set it up and configure it properly. More wasted time.
I'm going to be installing this at a datacenter. I ain't stuffing consumer products in that rack.
I didn't read this thread entirely but it appears you are looking for an enterprise level router for 10 meg to start, then scale to 100 meg or so?
Cisco 2800/3800 series or Juniper J2300 series would handle that kind of load just fine. With NAT, go 3800 on the Cisco just to be safe. I would give yourself more than a couple of days to configure though if you've never dealt with one of them before to be done properly. I have no idea why you be involving NAT though on the surface - perhaps you have good reasons but please tell me it has nothing to do with security - please ;)
--
Nexicom High Speed Internet - »www.nexicom.net/
reply
milnoc @ 27th Sep 07:18PM:
Re: Blade servers
All I want to do is restrict access to the internal network to authorized applications. The things that are configured in the port forwarding page of my office router are OpenVPN, FTP, VLC Media Player, and BitTorrent. The network at the datacenter will be configured in a similar manner.
NAT and port forwarding are one of the easiest and most effective ways I've found to protect my systems, and it can be done with a $50 router. The security has yet to be compromised both at home and at my office even after years of use.
I was hoping to find the same level of protection and flexibility for my datacenter equipment, but of course with better throughput and better reliability. But it sometimes seems as if it's mission impossible.
reply
pstewart @ 27th Sep 07:27PM:
Re: Blade servers
said by milnoc :
All I want to do is restrict access to the internal network to authorized applications. The things that are configured in the port forwarding page of my office router are OpenVPN, FTP, VLC Media Player, and BitTorrent. The network at the datacenter will be configured in a similar manner.
NAT and port forwarding are one of the easiest and most effective ways I've found to protect my systems, and it can be done with a $50 router. The security has yet to be compromised both at home and at my office even after years of use.
I was hoping to find the same level of protection and flexibility for my datacenter equipment, but of course with better throughput and better reliability. But it sometimes seems as if it's mission impossible.
NAT = security through obscurity. ;)
On commerical/enterprise level routers it's a lot more effecient to use access-lists or similar to only allow certain ports through to certain IP's. On a low traffic office environment it makes sense to use NAT but in the data center/service provider environment and higher traffic I prefer to avoid it at all costs.
--
Nexicom High Speed Internet - »www.nexicom.net/
reply
milnoc @ 27th Sep 07:29PM:
Re: Blade servers
So what do you use? How does your server farm interact with the Internet? Can you give examples of the type of IPs that you use?
reply
pstewart @ 27th Sep 07:36PM:
Re: Blade servers
said by milnoc :
So what do you use? How does your server farm interact with the Internet? Can you give examples of the type of IPs that you use?
We use public IP blocks for just about everything except our own internal networks (workstations, printers, etc).
So, one of our LAN segments is made up of Windows Servers for example - obviously we don't want very many ports open on these Windows servers so on our Cisco Firewalls (in our case) we simply tell it what ports on what IP's to permit:
xxx.xxx.xxx.22 - hostname1.domainname.net
Ports 25, 110, 80, 443 TCP
Repeated for each host.... these are only for inbound traffic to the server farms. Outbound from the server farms to the Internet is wide open at the moment in our case (soon to change).
Does that help explain what I mean? ;)
--
Nexicom High Speed Internet - »www.nexicom.net/
reply
milnoc @ 27th Sep 07:43PM:
Re: Blade servers
So I can do what I need to do with a Cisco 1800? My set-up is very much like yours, but highly simplified since it involves a small number of components. The servers can freely access the Internet. I just need to restrict access from the Internet to my servers only to the ports that have application servers running on them. The only time someone from the outside can have free run on the internal network is if they're connected to it via OpenVPN.
reply
pstewart @ 27th Sep 07:46PM:
Re: Blade servers
said by milnoc :
So I can do what I need to do with a Cisco 1800? My set-up is very much like yours, but highly simplified since it involves a small number of components. The servers can freely access the Internet. I just need to restrict access from the Internet to my servers only to the ports that have application servers running on them. The only time someone from the outside can have free run on the internal network is if they're connected to it via OpenVPN.
Yes, a Cisco 1800 will do the same thing as it's larger counterparts - just limited by throughput/CPU etc. For your 10 meg connection it would work just fine - as your traffic increases you'll have to swap it out for something larger.
--
Nexicom High Speed Internet - »www.nexicom.net/
reply
milnoc @ 27th Sep 08:02PM:
Re: Blade servers
It'll take a while before I reach 100 Mbps. And at under $900 Canadian for the 1811, it's a small price to pay for what I need it to do.
Thanks!
reply
milnoc @ 28th Sep 12:05PM:
Re: Blade servers
Unless a miracle happens in the next few minutes, the IBM BladeCenter deal is dead.
I found out why my supplier made so many mistakes with the order. It was the IBM Configurator software he was using. It allowed a disc array to be configured, but didn't include the SAS expansion cards that would allow the blades to actually access the disc array.
As for using ordinary SATA drives in the chassis, IBM will not honor the warranty on the entire set-up. They're extremely anal about you buying ONLY their equipment.
What's ironic is that I was planning to use Hitachi Deskstar drives in the chassis. Yes, the same hard drive company IBM sold to Hitachi many years ago. In fact, it's very possible IBM uses the same hard drives for their BladeCenter systems, but charges up to TEN times the street price of an ordinary SATA drive.
Twelve drives at 900 dollars each. The price of a used car. I'm not buying it. Especially the support and warranty excuses.
It's too bad. The system would have done the job perfectly.
reply
olebiker @ 28th Sep 02:15PM:
Re: Blade servers
Good to know that it is still the same company that I competed with in the 70's.
reply
jfmezei @ 28th Sep 03:54PM:
Re: Blade servers
>Good to know that it is still the same company that I competed with in
>the 70's.
Are you Gene Amdhal ?
BTW, every large remaining computer company is just as bad. Someone recently tried to evaluate some HP proliant server. It comes with a CD/DVD drive and its model number was given, but nowhere on the HP web site are the specs for that model available. And it is an HP branded model.
Turns out there is good reason for HP to not divulge the specs: it is sold on a current model, but has specs dating back from last century (it can read/write CDs, but only read DVDs).
In the end, the only way to be sure is to find and read the installation manuals (and the maintenance manuals if available).
reply
olebiker @ 28th Sep 04:48PM:
Re: Blade servers
said by jfmezei :
>Good to know that it is still the same company that I competed with in
>the 70's.
Are you Gene Amdhal ?
No good guess though. I knew many of their reps back then and was never too impressed with their own high opinion of themselves. I am sure that not one of them remains today though.
reply
milnoc @ 28th Sep 05:07PM:
Re: Blade servers
said by jfmezei :
In the end, the only way to be sure is to find and read the installation manuals (and the maintenance manuals if available).
I discovered the mistakes in the order by reading the S chassis' Redbook. It almost felt as if I knew more about the BladeCenter's capabilities than both the reseller and IBM!
I'm moving on to the Asus server components. I already have two of their RS100-X5/PI2 servers, and I'm very pleased with the results. I even installed sound cards in them! :) Unfortunately, there are no Linux drivers for the cards, so one server remains silent. :(
reply
anon @ 28th Sep 06:44PM:
Re: Blade servers
>What's ironic is that I was planning to use Hitachi Deskstar drives in the chassis.
Just hope you are buying enterprise grade (vs consumer grade) HD if there is such a classification for the Hitachi drives.
On Seagate drives, the enterprise grade is spec'ed for a higher duty cycle and may have different designs (likely in firmware) to handle the higher vibrations from having multiple drives mounted.
reply
twizlar @ 28th Sep 06:00PM:
Re: Blade servers
Take a look at the supermicro gear too.
reply
chronoss2009 @ 28th Sep 06:23PM:
Re: Blade servers
wait a minute your launching a CABLE channel ....um why...
id like to know what you mean about it being computer based OR are you meaning to do a IPTV via cable so why is this on a dsl forum.....
and while slash dot is often a place of funny and silly there was a neat article about a company selling there super computers for around 9 grand. the specs were quite impressive and scalable go have a look at there hardware area....forget the link
reply
milnoc @ 28th Sep 06:31PM:
Re: Blade servers
Why? Because I have a CRTC license approval that says I can. :)
Checking out the Supermicro stuff. I'm out of time so I might wait before studying that route. But I'll definitely take it into consideration. Those 2.5 inch hard drive systems will be terrific for the archives.
I've already priced the Asus systems. Asus RS120-E5/PA4, Quad-core Xeon X3220 2.4 GHz processor, 8 GB memory, 4 1 TB drives in a software RAID-5 array (3 TB total storage), $1,500 each not including shipping and taxes.
reply
milnoc @ 28th Sep 07:21PM:
Re: Blade servers
I'm not worried. I've used Hitachi drives before, some of them in constant use. I've yet to lose a single one.
(famous last words :) )
Checked out the Supermicro 5015B-MTB at Newegg. Wondering if I should get one of those along with one of the Asus boxes. They both use the exact same components.
reply
twizlar @ 28th Sep 07:38PM:
Re: Blade servers
Is there a reason you are using such old processors? You lose the ability to upgrade and the performance/power gains from the newer cpus.
--
Broadline Networks Inc.
reply
olebiker @ 28th Sep 07:56PM:
Re: Blade servers
said by milnoc :
I'm not worried. I've used Hitachi drives before, some of them in constant use. I've yet to lose a single one.
(famous last words :) )
Checked out the Supermicro 5015B-MTB at Newegg. Wondering if I should get one of those along with one of the Asus boxes. They both use the exact same components.
Had a TB Hitachi HD fail on the weekend. It was in service 30 days. Lost 500GB data. Not too impressed but it is a first so I replaced it with another Hitachi. Seagate consumer drives are not very good.
reply
twizlar @ 28th Sep 08:04PM:
Re: Blade servers
Their latest revisions are actually great, what you need to watch out for are the 7200.11 500Gb-1TB models.
--
Broadline Networks Inc.
reply
olebiker @ 28th Sep 08:14PM:
Re: Blade servers
said by twizlar :
Their latest revisions are actually great, what you need to watch out for are the 7200.11 500Gb-1TB models.
Thanks perhaps my info is out of date already.
reply
milnoc @ 28th Sep 09:34PM:
Re: Blade servers
said by twizlar :
Is there a reason you are using such old processors? You lose the ability to upgrade and the performance/power gains from the newer cpus.
Until I can find an adequate replacement for the botched BladeCenter experience and catch up on all that lost time, I need to spend the minimum required to get a basic broadcast going ASAP. I'll then have the time to do all the necessary research and find an adequate and upgradeable replacement that will stick around for years.
The servers I purchase today will be recycled for other uses later on. They're still more than powerful enough to do all kinds of cool things with them. Don't sell them short.
BTW, the drives I'm planning to purchase are the Hitachi 7K1000.B series. Another option is the Western Digital WD1001FALS Caviar Black series. If anyone can tell me which one is more reliable, please let me know before they crash. :)
I can even get both, one set for each server for multi-vendor redundancy, which is why I'm also considering substituting one of the Asus servers with a Supermicro server. The cost is about the same.
reply
Guspaz @ 28th Sep 09:45PM:
Re: Blade servers
If you build a proper RAID array, you shouldn't need to worry so much about which manufacturer's drives you pick. Although, if you want to be safer, RAID-6 is nice. So is RAIDZ2.
RAID10 is gaining favour, but it has a major flaw. Any one disk can fail recoverably, but after that, of the remaining two disks, only two of them can fail recoverably. There's a 33% chance that the second disk failure will nuke the array. With parity-based stuff like RAIDZ2 and RAID-6, any two disks can fail recoverably.
reply
milnoc @ 28th Sep 10:04PM:
Re: Blade servers
Already on it. But for a recoverable two drive failure set-up to be worthwhile economically, the array needs to be reasonably big. For now, I'll work with two separate arrays on two separate servers. This way, if one array dies (or the server dies), the other will keep working.
This whole process is beginning to give me a headache.
reply
anon @ 28th Sep 11:02PM:
Re: Blade servers
Because of the storage size of the drives, RAID5+1 is highly recommended. There is a non-zero chance that during the sustain disk activities in rebuilding the RAID might actually encounter a non-recoverable read error as you disk size approaches the same order of magnitude as the error rate.
»blogs.zdnet.com/storage/?p=162
If only there is a way that you could setup the RAID across your servers such that they are complementary ECC set in the RAID. i.e. they logically form a higher RAID order. Something similar to those .par2 files on usenet where you download as much as needed to fix a bad dataset in that you swap redundant drives from one to the other.
reply
valunthar @ 29th Sep 12:15AM:
Re: Blade servers
Hmm, you might be able to pull that off using ESATA and nested raid levels. If it works it'd be a pretty complicated setup though.
reply
Guspaz @ 29th Sep 12:35AM:
Re: Blade servers
RAID6 addresses the issue of read errors on rebuild without wasting space like RAID5+1 does. The chances of the exact same sector being bad on two disks in a RAID-6 array are astronomical, handily solving the reliability issues of RAID5.
reply
InvalidError @ 29th Sep 01:33AM:
Re: Blade servers
said by An_Onymous :
There is a non-zero chance that during the sustain disk activities in rebuilding the RAID might actually encounter a non-recoverable read error as you disk size approaches the same order of magnitude as the error rate.
RAID-6 is essentially ECC. If you have N "parity" drives, RAID6 can reliably detect N errors, correct N/2 errors and compensate for up to N drive failures.
RAID1/3/4/5/10 can only detect a mismatch between drives on soft errors but has no idea which drive contains the correct data unless the drive failed. For mission-critical, having three 'parity' drives in a RAID6 array guarantees that a single drive failure will not compromise the system's error detection and correction capabilities until/unless a second drive failure occurs before the reconstruction is completed.
RAID6 does not scale nearly as well performance-wise but it does enable much more resilient array configurations.
reply
milnoc @ 29th Sep 09:17AM:
Re: Blade servers
I'll be ordering all of the equipment today. If anyone has any caveats on my choice of hard drives (Hitachi 7K1000.B and WD Caviar Black) and servers (Asus RS120-E5/PA4 and Supermicro 5015B-MTB), please let me know before lunch.
reply
twizlar @ 29th Sep 10:36AM:
Re: Blade servers
Where you getting them from? Just looking around you can get much better hardware for the same/less money. Both of those barebone kits are quite outdated. For the harddrives, the Black edition WD's are pretty good, they are basically a Raid Edition drive.
--
Broadline Networks Inc.
reply
milnoc @ 29th Sep 10:57AM:
Re: Blade servers
The Asus will be about $600 CAD, while the Supermicro 5015B-MTB will be about $700 CAD. The processors are $190 each. Memory (from Crucial -- won't compromise there) $360 USD for both servers.
Remember, these are servers with four hot-swap drive bays in them, each costing on average $1,500. If you know where I can find better for the same amount of money, please list me some model numbers.
I've been using Shopbot.ca to help me find the best deals. I've also been cross-checking with the original manufacturers sites so that I'd be buying from authorized dealers. The dealers I've found for everything are newegg.ca, Agile Electronics, and TigerDirect.
reply
milnoc @ 29th Sep 11:02AM:
Re: Blade servers
Oh. Another reason to use older equipment. They have a proven track record. New equipment? Not sure I'd want to risk it.
reply
Guspaz @ 29th Sep 11:15AM:
Re: Blade servers
said by InvalidError :said by An_Onymous :
There is a non-zero chance that during the sustain disk activities in rebuilding the RAID might actually encounter a non-recoverable read error as you disk size approaches the same order of magnitude as the error rate.
RAID-6 is essentially ECC. If you have N "parity" drives, RAID6 can reliably detect N errors, correct N/2 errors and compensate for up to N drive failures.
RAID1/3/4/5/10 can only detect a mismatch between drives on soft errors but has no idea which drive contains the correct data unless the drive failed. For mission-critical, having three 'parity' drives in a RAID6 array guarantees that a single drive failure will not compromise the system's error detection and correction capabilities until/unless a second drive failure occurs before the reconstruction is completed.
RAID6 does not scale nearly as well performance-wise but it does enable much more resilient array configurations.
Errm, no, it isn't... RAID 6 is identical to RAID 5 with the addition of double parity. It can survive two drive failures to RAID 5's one. It does not support arbitrary numbers of parity drives. You're describing something entirely different.
reply
milnoc @ 29th Sep 11:21AM:
Re: Blade servers
Guspaz, any opinions on RAID-1E? Although it can't survive two adjacent drives failing, its low overhead and cost makes it seem like an interesting alternative.
reply
milnoc @ 29th Sep 11:57AM:
Re: Blade servers
Interesting turn of events.
My supplier called. Apparently, IBM wants to know more about my channel. They may be interested in proposing a better deal.
I can put the ordering of the alternate equipment on hold for a couple of days. Delivery of that equipment can be accomplished within a week, and I can set everything up during the Thanksgiving holiday if necessary. So if IBM is still too expensive, or the delivery date is too far ahead, I still have my fall-back plan.
I would prefer the IBM equipment despite the extra cost. The extra redundancy in the S chassis makes it an ideal candidate for my channel. The problem with the individual component set-up is that it's not as redundant as I would like it to be. Too many individual failures can still bring down half of the entire system.
reply
anon @ 29th Sep 12:14PM:
Re: Blade servers
Just out of observation of highly available equipments. In general, they tend to have put redundancy on the critical components (HDD, power supply, fans, network/fibric, ECC RAM) that could easily have some form of recovery/graceful degrade.
- Are there any redundancy on your power supply?
i.e. Any protection against if the power supply failures? (not power failure as you would have some protections at the data center.)
- Are there any redundancy on your ventilation?
i.e. extra fans that kicks in or have the extra capacity to cool the box even when a fan fails.
reply
milnoc @ 29th Sep 12:24PM:
Re: Blade servers
Four power supplies, four large ventilation fans, all hot-swappable. Even the hard drive controllers and network modules are hot-swappable.
The best way to kill off an IBM BladeCenter in one shot is to use an axe! :D
reply
mlerner @ 29th Sep 12:25PM:
Re: Blade servers
After that experience I would have cut all ties from IBM and the supplier. To me that kind of service is completely unacceptable. They better offer a significant discount.
reply
milnoc @ 29th Sep 12:31PM:
Re: Blade servers
That's why I'm giving them one last chance. It's put-up-or-shut-up time.
One thing's for certain. If I do get the BladeCenter, I'll review it for my YouTube channel. There's absolutely no impartial reviews of this gear anywhere to be found.
reply
Guspaz @ 29th Sep 01:56PM:
Re: Blade servers
said by milnoc :
Guspaz, any opinions on RAID-1E? Although it can't survive two adjacent drives failing, its low overhead and cost makes it seem like an interesting alternative.
I'm not sure what the advantage of RAID-1E is over RAID-1 unless you have an odd number of drives.
RAID-5 and RAID-6 have poor write performance, but good read performance.
Which level of RAID to choose all comes down to your requirements:
1) How important is redundancy/recoverability?
2) How important is storage capacity/cost?
3) How important is read performance?
4) How important is write performance?
I'd argue that starting at 4 and ending at maybe 6 drives, RAID-6 or any other double-parity system (such as RAIDZ2) is probably going to provide the greatest degree of reliability. At three drives, both RAID-5 and RAID-1E offer the same level of reliability (can tolerate only one drive failure), but RAID-5 wastes less space and RAID-1E is faster for writes/rebuilds. At two drives, RAID-1 is the only option.
I don't know, as I said, it really comes down to your requirements.
reply
jfmezei @ 29th Sep 02:14PM:
Re: Blade servers
If you choose anything more complex that mirroring/shadowing (Raid 1), you need to consider the case where the controllers fail.
You will need to have the same controller models (and probably firmware) and be able to reload the exact same controller config in order to be able to access the data on your drives.
Today, when you install your system, you don't see this as a problem. But 3 years down the road, when that controller is no longer on the market or has evolved with much newer firmware, you may find yourself in a situation where you can't access yoru raid array even though the data on the drive is still there.
reply
milnoc @ 29th Sep 02:46PM:
Re: Blade servers
I'm considering using a software RAID approach since my operating system will be Linux. I won't have to worry about the technology becoming outdated. The only issue is that there doesn't appear to be a RAID-1E implementation under Linux. That could have struck the right balance between protection and performance.
reply
twizlar @ 29th Sep 02:46PM:
Re: Blade servers
Realistically the write penalties often associated with RAID5 aren't much of an issue anymore with the newer, faster drives. RAID5 or RAID6 are probably your best options
--
Broadline Networks Inc.
reply
Guspaz @ 29th Sep 05:05PM:
Re: Blade servers
Linux can do the same thing as RAID-1E on 3 disks. But as I said, I'm not sure that I really see any advantages to RAID-1E on even numbers of disks.
reply
milnoc @ 29th Sep 05:27PM:
Re: Blade servers
Try six discs. :)
Having an odd or even number of discs is of no importance for a RAID-1E system. The biggest potential advantage with the format however is that it requires very little computing power to make it work. There's no parity math involved whatsoever, so it should be very efficient even when managed by the operating system itself. This could save you a fortune when compared with the purchasing cost of a hardware RAID-5 or RAID-6 solution ($6,000 on the BladeCenter alone).
RAID-1E's biggest drawbacks however are that it cuts your total storage size by half. And if two adjacent drives go down, you lose the entire array.
When I get an external archive system, it'll probably be one of those multi-drive units that's actually a server in itself. I'll just load Linux on it and run it as a dedicated RAID-6 or equivalent NAS. Supermicro has models that can hold 16 3.5 inch hard drives, or 24 2.5 inch hard drives!
reply
jfmezei @ 29th Sep 08:57PM:
Re: Blade servers
Raid1 does provide advantages (if the controller is done right) for reads as controllers can distribute read requests on multiple disks so there is less queueing for read IOs.
reply
milnoc @ 30th Sep 05:54PM:
Re: Blade servers
I'm expecting a new quote from my supplier along with the delivery dates. An acceptable compromise may have been reached by all parties.
If the system is installed, I'll later find an IBM engineer who isn't part of the sales and marketing department, and propose a challenge to them. It could produce some interesting results.
I've been checking out other equipment for the last couple of days (Supermicro, Asus, a few other). I have to be honest that the overall cost and maintenance headaches involved in creating a redundant system out of a ton of components from different manufacturers isn't very reassuring. It won't be very reassuring for the cable companies either when they ask me what I have in my equipment rack. This is the main reason why I'm still pursuing the IBM option. It's just as much a question of marketing as it is of reliability.
More to come.
reply
milnoc @ 1st Oct 10:09PM:
Re: Blade servers
The IBM deal is dead. They screwed up the order once again. The drives in the storage bays would have been inaccessible due to an incompatibility between the chosen drives and the controllers. It was clearly indicated in their own Redbook!
I might as well order those Asus and Supermicro servers tomorrow. It's the best I can do until I can find the time to research and purchase better quality equipment. Here's hoping both servers don't crash at the same time once the channel is on the air.
I'm starting to wonder if I should contact a lawyer about all of this.
reply
twizlar @ 1st Oct 10:41PM:
Re: Blade servers
said by milnoc :
The IBM deal is dead. They screwed up the order once again. The drives in the storage bays would have been inaccessible due to an incompatibility between the chosen drives and the controllers. It was clearly indicated in their own Redbook!
I might as well order those Asus and Supermicro servers tomorrow. It's the best I can do until I can find the time to research and purchase better quality equipment. Here's hoping both servers don't crash at the same time once the channel is on the air.
I'm starting to wonder if I should contact a lawyer Network Guy about all of this.
--
Broadline Networks Inc.
reply
milnoc @ 1st Oct 11:01PM:
Re: Blade servers
How will a network guy help? The network is not the issue here. It's all about reliability and redundancy.
reply
twizlar @ 1st Oct 11:19PM:
Re: Blade servers
I mean someone that can spec you the proper hardware from a reputable vendor.
--
Broadline Networks Inc.
reply
milnoc @ 2nd Oct 06:56AM:
Re: Blade servers
How can I be certain that person isn't in collusion with a particular vendor?
reply
Guspaz @ 2nd Oct 10:44AM:
Re: Blade servers
Find somebody who hates that vendor.
reply
milnoc @ 2nd Oct 11:01AM:
Re: Blade servers
I just have to look in the mirror for that! :)
reply
milnoc @ 3rd Oct 12:41AM:
Re: Blade servers
Quick follow-up. I received an email from my former supplier indicating that the drive incompatibility has been resolved with a firmware update.
This answer doesn't inspire much confidence. Do I really want to take a chance with a new firmware that might or might not work properly? I'm not interested in losing all of my data due to an unforeseen "bug."
Plus, the IBM rep. had "upgraded" the server specifications when it wasn't requested, needlessly increasing the price by about $1,500.
I am truly fed up with them.
I just checked out the Dell PowerEdge R410. The hard drive prices are just as outrageous as IBM. Even worse, there are reports of major problems installing Ubuntu 9.04, which is what I'm targeting.
I think I'll stick with Asus. Two barebones servers and three netbooks later, I've yet to kill any of them off with my incessant upgrades. They're still chugging along. I've also read comments on successful software RAID tests performed on their machines i.e. pull out a drive, reinsert it, and it gets rebuilt automatically.
Why are the major manufacturers so difficult in their specifications, and the low-key manufacturers so flexible and operational? It all seems like one giant con job.
reply
anon @ 3rd Oct 08:18AM:
Re: Blade servers
>Why are the major manufacturers so difficult in their specifications, and the low-key manufacturers so flexible and operational? It all seems like one giant con job.
You are comparing 2 different market sectors. IBM is coming in from the high availability market...
To make a PC mass market level into enterprise or server grade needs quite a bit of engineering. Their internal process can easily inflates the cost by a few time just because of anal paper pushing exercises and then there is the cut from their marketing/sales guys. Where else are they making their NRE investment back except to sell you $1000 HD?
Your PC motherboard supplier is used to having to deal with 2 or 3 product rev. and variations and product lifetime of less than 1 year and also undercutting from other brands flooding the market. They have their internal process optimized for paper thin margins.
reply
olebiker @ 3rd Oct 01:26AM:
Re: Blade servers
said by milnoc :
Quick follow-up. I received an email from my former supplier indicating that the drive incompatibility has been resolved with a firmware update.
This answer doesn't inspire much confidence. Do I really want to take a chance with a new firmware that might or might not work properly? I'm not interested in losing all of my data due to an unforeseen "bug."
Plus, the IBM rep. had "upgraded" the server specifications when it wasn't requested, needlessly increasing the price by about $1,500.
I am truly fed up with them.
I just checked out the Dell PowerEdge R410. The hard drive prices are just as outrageous as IBM. Even worse, there are reports of major problems installing Ubuntu 9.04, which is what I'm targeting.
I think I'll stick with Asus. Two barebones servers and three netbooks later, I've yet to kill any of them off with my incessant upgrades. They're still chugging along. I've also read comments on successful software RAID tests performed on their machines i.e. pull out a drive, reinsert it, and it gets rebuilt automatically.
Why are the major manufacturers so difficult in their specifications, and the low-key manufacturers so flexible and operational? It all seems like one giant con job.
It has always been that way the smaller guys have to try harder and do a better job to grow.
reply
jfmezei @ 3rd Oct 02:04AM:
Re: Blade servers
> It has always been that way the smaller guys have to try harder and do a better job to grow.
Another way to see it is that the bigger guys come out with systems that are compromises due to internal politics and unhealthy urge for proprietary designs, and the smaller guys can then look at the flaws of the bigger guy's offerings and come out wth better/simpler more open designs.
Also, the smaller guys have fewer resources to design their own proprietary systems so they have more incentive to use off the shelf components which allow cutsomers to find lower priced disks etc.
reply
olebiker @ 3rd Oct 02:14AM:
Re: Blade servers
said by jfmezei :
> It has always been that way the smaller guys have to try harder and do a better job to grow.
Another way to see it is that the bigger guys come out with systems that are compromises due to internal politics and unhealthy urge for proprietary designs, and the smaller guys can then look at the flaws of the bigger guy's offerings and come out wth better/simpler more open designs.
Also, the smaller guys have fewer resources to design their own proprietary systems so they have more incentive to use off the shelf components which allow cutsomers to find lower priced disks etc.
Good points.
reply
jfmezei @ 3rd Oct 02:35AM:
Re: Blade servers
A good example of this was Digital in the early 1990s. Digital designed "DSSI" which was electrically almost identical to SCSI-1 but with different controllers that removed some of the restrictiors of the the SCSI protocol as it was back then. Namely ability to have multiple hosts connected to the same bus, and ability for hosts to talk to each other via that bus.
This allowed multiple nodes to access the same physical disks as well as use DSSI to talk to each other (with ethernet as a fallback). (Later on, SCSI did gain the ability to support multiple controllers on the same bus so the raison d'être for DSSI ceased.
However, during the time DSSI and SCSI co-existed on the market, DEC's DSSI drives were much more expensive than standard SCSI drives which others were using. It did give additional capabilities (and their controllers allowed you to "telnet" to the disk to access it config, drive statistics, utilities and error logs).
And back then, the smaller computer makers like Sun and HP didn't have the resources to design sucgh proprietary subsystems so they tended to use off the shelf simpler stuff.
DEC's DSSI didn't last very long. But in many ways, it shows what storage subsystems are today. A few "rich" companies design expensive storage systems that are proprietary and give you a number of features not available with off the shelf cheaper systems.
IBM is one such company that does have money to design higher "quality" stuff, but you are then restricted to IBM approved devices that interface to the proprietary additions that IBM made in order to add a few bells and whistles.
And this is especially true of all blade systems (IBM, HP and others) since those are pitched as "industry standard" servers, but are made up of extremelly proprietary components that are sold with a premium which pleases accounts to no end since those love the hgher margins that blade sales produce.
The one area where such proprietary systems are "needed" is fault tolerant systems such as those built by Tandem (now unfortunatly part of HP). They need specialised components that have fault detection hardware and interface with the special controlers that have hardware failover when a fault is detected. And those components are also designed and tested with high MTBF and designed for continuous operation. And you pay a hefty premium for this and are limited to only Tandem hardware becfause of the very proprietary hardware interfaces.
reply
mlerner @ 3rd Oct 08:58AM:
Re: Blade servers
said by milnoc :
Quick follow-up. I received an email from my former supplier indicating that the drive incompatibility has been resolved with a firmware update.
This answer doesn't inspire much confidence. Do I really want to take a chance with a new firmware that might or might not work properly? I'm not interested in losing all of my data due to an unforeseen "bug."
Tell them to shove it and you'll take your business elsewhere. The fact they don't inspire confidence in what they're selling is a dead giveaway that you can't trust them with that amount of cash. Considering this is production hardware, do you really want to trust a vendor who doesn't know what they're selling? This is enterprise hardware, you expect top notch service.
reply
milnoc @ 3rd Oct 09:13AM:
Re: Blade servers
That move away from established standards, and charging ten times the street price for a hard drive, is what's driving me away from the large companies more than anything else. What does this say about the large corporations when you can't even dump any old Linux distro on their computers, but the low key manufacturers that use off-the-shelf components have absolutely no trouble handling nearly any OS you throw at them?
And may I add that for companies like Asus and Supermicro, once their products have been installed at a datacenter and have survived for the first couple of months, they seem to work fine for years. I'm having trouble finding any significant complaints about product failures with these companies.
I can understand if a enterprise grade computer costs three times the price. There is after all a lot of engineering built into them, and I was ready to pay that price. But ten times the street price for a hard drive? I just don't buy it! On the Dell PowerEdge R410 alone, the cost of the four 1 TB hard drives exceeds the price of the server itself, even when fully equipped!
No! I will NOT support Bose and Monster Cable style marketing tactics! It's time to move on and get this broadcast system running once and for all.
reply
anon @ 3rd Oct 10:22AM:
Re: Blade servers
>That move away from established standards, and charging ten times the street price for a hard drive, is what's driving me away from the large companies more than anything else.
I don't even know why companies wants to supply a real HDD in their system other than the mounting kit. I can understand that from support point of view and marketing to sell a complete system. Putting in a HDD add so much overhead that it is not funny.
Every time a HDD changes rev. or models which tends to happen more than once a year. There is a bit of system level qualification work/testing needed to be done again. There is also the FCC/CSA/UL BS, even if it is a paper exercise there is a dollar figure and time factor for the external compliance labs. Added to that, there is the new part#, manufacturing inventory, finished goods stock, and marketing BS which adds overhead again.
This is one of the many reasons why the peripheral (memory/HDD) prices from vendors are out of wack with the DYI clone shop down the street. From the chipset and parts point of view, the lifetime cycle is a longer so rest of system is a bit closer to your expected mark up.
I remember we had the Tektronic guy coming to "install" memory expansion for a logic analyzer. We were all keen as engineers trying to see what's inside the box and what the $10k memory looks like. We were very disappointed to see the guy comes in with a CD to change the system configuration unlock the memory. ;(
As for monster cable, it is not like they have done as much paperwork or engineering testings. It is in the snake oil business and not in engineering business. I usually buy the cheapest one if I don't have the connectors. If it breaks, I cut open the molded plastic, fix it up with better wires and glue it back.
reply
milnoc @ 3rd Oct 10:16AM:
Re: Blade servers
said by mlerner :
Tell them to shove it and you'll take your business elsewhere. The fact they don't inspire confidence in what they're selling is a dead giveaway that you can't trust them with that amount of cash. Considering this is production hardware, do you really want to trust a vendor who doesn't know what they're selling? This is enterprise hardware, you expect top notch service.
Isn't it shenanigans like these that have badly hurt IBM in the past? They're behaving as if they're still being run by an "old boys" club.
reply
mlerner @ 3rd Oct 10:24AM:
Re: Blade servers
said by milnoc :said by mlerner :
Tell them to shove it and you'll take your business elsewhere. The fact they don't inspire confidence in what they're selling is a dead giveaway that you can't trust them with that amount of cash. Considering this is production hardware, do you really want to trust a vendor who doesn't know what they're selling? This is enterprise hardware, you expect top notch service.
Isn't it shenanigans like these that have badly hurt IBM in the past? They're behaving as if they're still being run by an "old boys" club.
It is an old boys club. Just the fact that IBM is pricing like they did in the old days is ridiculous and they're trying to stop you from using your own drives, nothing but a cash grab. These days hard drives are all standard now. If they're trying to charge you the extra bucks, the least they can do is give you 10 star service.
If I were you I'd be filing a complaint and buying elsewhere. You are starting a business and they are essentially costing you money for every delay. That does not fly in my book.
reply
milnoc @ 3rd Oct 11:06AM:
Re: Blade servers
$1,865 in operating expenses alone. Throw in my time if I were paid, and the total comes to $7,000. That's the cost of about four bargain off-the-shelf rackmount servers with four drive RAID arrays.
I'll never see that money again.
reply
jfmezei @ 3rd Oct 03:28PM:
Re: Blade servers
IBM prices are NOWHERE near like they used to be. IBM has become far more competitive than it used to be. Perhaps they haven't moved enough, but they did move plenty.
In terms of drives that are more expensive. This is similar to Apple's mentality. It is all about support. When your hardware supports only a narrow set of components, it is far easier to support it, and far easier to qualify the components to ensure they work properly.
When Digital in its last years, started to move to industry standard components, they found they had a lot of trouble getting "anything" to work perfectly. Even SCSI disks had quirks because the SCI protocol had grey areas where different disks behaved differently. For instance, the VMS OS expected the disks to report any errors they encountered, but some SCSI drives did the remapping of blocks internally and didn't report the errors. VMS system managers would look at the OS level error logs and see no problems when in fact the disks were experiencing problems.
You'll find that when you go to commercial storage arrays, you can only plug in proprietary disks embedded into the proprietary carriers. And yes, you pay much more. A small part is due to the higehr costs of the disk, but another is just the fact that markleting has decided to set the price to "what the market will bear" instead of "how low can our price be".
Also, storage system vendors selects only hard drives that have "enterprise" MTBF numbers, not the elcheapo drives you find at your local computer shop. In many ways, those are identical, but the expensive ones got extra testing to ensure they have no flaws. (for those old enough to remember floppy diskettes, there were single density and double density diskettes, and they were basically identical except for a notch on the enveloppe. The double dnesity ones were tested for double density and the single density ones were not. Chances were high that by puching that notch, you coudl convert the single density into a double one.
reply
milnoc @ 3rd Oct 04:29PM:
Re: Blade servers
Unfortunately, the pricing of IBM hard drives are such that they're simply not believable. The rest of the gear is about two to three times more expensive than conventional gear, which makes sense considering the technology and the work they put into the design.
But TEN TIMES for an IBM branded hard drive? And in a system that can hold TWELVE of them not including the hard drives in the servers themselves? I'm not buying it.
I don't give a rat's ass what "the market can bear." I care about what my business can bear. And ten grand for a dozen hard drives, or even five grand for six of them, is simply ludicrous.
Just checked out Intel's offerings. They do have a "modular" system, but their Web site is a complete mess. I can't extract any useful information from it.
reply
invictive @ 3rd Oct 06:23PM:
Re: Blade servers
This may work for your storage array, it supports 24xSATA drives: »www.hardforum.com/showpost.php?p···ount=220
Supermicro is highquality and very reliable. You may want to look at an Areca raid card as well.
reply
milnoc @ 4th Oct 09:16AM:
Re: Blade servers
I did check out Supermicro. Not bad. I'll definitely consider them for storage.
I'll need to expand my rack to accommodate two of those beasts. Each one is probably 5U in height!
I've been looking at the Asus servers from their US site instead of the Canadian site. They have new models not announced on the Canadian site, more specifically the RS700-E6/RS4 with four drive bays and redundant power supplies. Twizlar is right about not going with equipment that's too old. The RS120 series may be just on the verge of usability.
There's also the RS700D-E6/PS8 which has two computers in it and can handle eight 2.5 inch hard drives, all in a 1U package. I could get that when I need more processing power for the broadcast engine. That, or the 2U high Supermicro 6026TT-BTF which has four computers in it.
However, I'm a little worried about cooling with both of those models. Four quad-core processors in a 1U space? That's a mighty powerful furnace! :)
reply
Guspaz @ 4th Oct 03:41PM:
Re: Blade servers
eight 2.5" drives in a 1U package isn't very much data, only 4TB. You can do that with two 3.5" drives. So that's not very optimal for data density.
reply
milnoc @ 4th Oct 06:16PM:
Re: Blade servers
It's not for data. It's to supply the extra broadcast feeds I'll eventually need, and still have a RAID set-up for the computers. The current programming files will remain stored on the two 3.5" drive servers and be accessible on the network.
reply
Guspaz @ 5th Oct 01:12AM:
Re: Blade servers
If raw throughput is your goal, 2.5" drives are not necessarily the most economical approach to get that. They have a much lower sustained transfer rate in general due to their smaller platters.
Case in point: the 300GB VelociRaptor, a 10K RPM 2.5" drive, has a similar average sustained throughput of a cheap 7200RPM drive.
If high IOPS is your goal, Intel's enterprise SSD drives are cheaper to hit the same IOPS than 15K RPM drives; one x25-e has a similar IOPS performance to eighteen 15K RPM drives in RAID.
reply
invictive @ 5th Oct 01:22AM:
Re: Blade servers
The case I posted above is 4U, so its pretty good density.
I would not be concerned about cooling with the Supermicro boards/cases, everything is designed to work together and they have very good cooling ducts. They usually use Sanyo Denki San Ace fans, which are the some of, if not the best. They are loud as hell but push 100+CFM.
reply
twizlar @ 5th Oct 01:22AM:
Re: Blade servers
Not really true anymore, the new seagate 2.5" SAS drives absolutely scream. They beat most of the other 2.5" and 3.5" drives. Where the new 2.5" drives really dominate is IOPS and throughput is almost on par as well.
--
Broadline Networks Inc.
reply
milnoc @ 5th Oct 03:01AM:
Re: Blade servers
This might sound surprising, but the entire system does NOT require a high IOPS. The file repositories will be used to store and distribute video files which will be streamed to cable companies, and not to the general public. The data throughput required to accomplish this is much lower than expected, especially if I deal only with raw feeds and perform no transcoding whatsoever.
As for the drives holding the OS, the entire system will be running off Linux. Based on my tests, the resource requirements for Linux are so low, they're laughable. In fact, I'm planning to swap the Cisco router for a GigE switch and a Supermicro 5015A-H Atom based rackmount server loaded with SmoothWall Express 3.0. I'll save $200-$300, and have more power and features in the network set-up.
When the system becomes mission critical, I'll double up on the network components and get a second ISP. This will ensure full 1+1 redundancy in the entire system.
I'll get this system to work! :)
reply
Guspaz @ 5th Oct 03:49AM:
Re: Blade servers
said by twizlar :
Not really true anymore, the new seagate 2.5" SAS drives absolutely scream. They beat most of the other 2.5" and 3.5" drives. Where the new 2.5" drives really dominate is IOPS and throughput is almost on par as well.
The new Seagate 2.5" SAS (such as the Savvio 15K.2) drives still get embarrassed by even consumer-level SSDs (which are also 2.5") in terms of IOPS. And their sustained transfer rate (Seagate's claims are 122-160MB/s) are matched by far cheaper 3.5" drives. And trounced by SSDs.
If you're looking for speed, magnetic is a silly place to look. And the worst part? Savvio drives cost more than Intel's x25-m at the same capacity, despite being ridiculously slower!
Admittedly, the x25-e is a more fair comparison. Even though the x25-m destroys the Savvio, it's a consumer-level drive.
The x25-e, which is about an order of magnitude faster than the Savvio, costs three times as much. Still a win if you care about IOPS/$.
reply
milnoc @ 5th Oct 12:07PM:
Re: Blade servers
Just thought of something. What if I used the Supermicro 5015A-H as the cable company broadcast client? One poster a while back pointed out that broadcasters would prefer a rackmount computer over an Eee PC. At a cost of about $450 each including processor, memory, hard drive and sound card (PCI or USB), it can work!
Imagine all the possibilities with this tiny box. Supermicro may have a potential hit on their hands.
5015A-H |
TSI Gabe @ 5th Oct 01:05PM:
Re: Blade servers
Just as a word of caution, almost every time we've ordered something from supermicro, it's taken them well over 2 months to show up at our door.
reply
milnoc @ 5th Oct 01:19PM:
Re: Blade servers
I've ordered it from NCIX. Their system reports it as being in stock. We'll see how it goes.
reply
twizlar @ 5th Oct 01:55PM:
Re: Blade servers
said by TSI Gabe :
Just as a word of caution, almost every time we've ordered something from supermicro, it's taken them well over 2 months to show up at our door.
They are quite slow, I don't think anyone that uses them regularly actually orders directly from them for timed shipments.
--
Broadline Networks Inc.
reply
Guspaz @ 5th Oct 05:08PM:
Re: Blade servers
That certainly is minneh, but that doesn't count for much in a rack unless you intend to mount two of them back-to-front.
reply
milnoc @ 5th Oct 05:41PM:
Re: Blade servers
The one I ordered will be put on the same shelf as the switch. Server at the front, switch at the back. There will be a huge gap between the two.
reply
milnoc @ 8th Oct 12:57PM:
Re: Blade servers
And the troubles keep on happening...
I've ordered components from TigerDirect, NewEgg, NCIX, DirectDial, and Crucial.
The servers were ordered from Newegg on Monday. But due to worrying delays in shipping the product out, I had to cancel the order, and order the servers from DirectDial instead on Tuesday. Both servers were delivered on time on Wednesday.
Crucial memory order placed on Monday, shipped on Tuesday, received on Wednesday. No problems there.
The NCIX order placed on Monday included two processors and the Supermicro Atom based server. The server was back-ordered (I already knew that), so they shipped the processors ahead of the server on Tuesday. I received them on Wednesday. I have a temporary substitute for the Supermicro server (my Eee PC 900), so no problems there. I can wait a while.
TigerDirect however is becoming a major pain. Order placed on Monday with UPS World Ease shipping, shipped very late on Monday. Because of the time, I was expecting the shipment on Wednesday along with everything else.
First problem with TigerDirect. The order got stuck in Hamilton due to a customs SNAFU which I suspect wasn't actually related to the customs office.
Second problem with TigerDirect. The order was short four hard drives. I only have enough matching hard drives to complete work on a single server, with no backup drive if one of the drives in the RAID array is DOA. The other server will have to wait.
TigerDirect is checking out the matter now to see what happened. But if they can't respond to me before 3 PM, I'll have to order the missing drives from another supplier that can deliver them overnight, or find a local shop that has them in stock.
Very annoying, especially since a long weekend is coming up, and I wanted to finish everything before the installation at the datacenter next week. Still, this is a MINOR delay in an order process that was completed within a week! This is much, MUCH better than the time that was wasted trying to get the IBM gear.
BTW, the Asus RS700-E6/RS4 servers are beautiful! It's a cable-free design that's very well built. Only time will tell if it's reliable enough for broadcast purposes. But if my current experience with Asus products are any indication, I should be okay.
More updates to come!
reply
Guspaz @ 8th Oct 02:08PM:
Re: Blade servers
What kind of drives are missing? Hard disks aren't exactly hard to find at decent prices in Montreal. PC Cyber generally has good prices, although they probably don't stock many enterprise drives.
reply
milnoc @ 8th Oct 02:30PM:
Re: Blade servers
Hitachi Deskstar 7K1000.B (1 missing), Western Digital Black WD1001FALS (3 missing). I'm using drives from two different vendors in case one vendor has a bad batch of drives.
I can get a couple of WD drives from Microbytes. They're just around the corner. The only annoyance is that they're almost $20 extra each, but at least there's no shipping costs, and I can get them NOW.
I'll wait for delivery of the on-line order for the fifth drive of each vendor. They'll be needed only if any of the other drives fail in their respective RAID arrays.
reply
olebiker @ 8th Oct 03:40PM:
Re: Blade servers
Leaving the frustration aside, you must be feeling somewhat excited at seeing things starting to roll in.
reply
milnoc @ 8th Oct 03:42PM:
Re: Blade servers
It is! Just like Christmas when I was a kid when my mom & dad would try to buy my love with a ton of crap! :D
reply
Guspaz @ 8th Oct 03:51PM:
Re: Blade servers
PC Cyber (corner of St-Catherine and Guy):
Hitachi Deskstar 7K1000.B: $89.77 ($129 at Microbytes, 4 in stock in Montreal)
Western Digital Black WD1001FALS: $99.77 ($119 at Microbytes, 0 in stock in Montreal)
Total estimated savings buying at PC Cyber: $96.92
Only naggling thing is that the WD isn't in stock in Montreal (unless you don't mind mixing brands), but PC Cyber will move stock around for you if you'd like. But then you need to wait (they have two shipments to Montreal a week).
But yeah, if they have it at PC Cyber, it's almost 100% guaranteed to cost a lot less than Micro Bytes.
The main reason is that the guy who runs Micro Bytes charges a percentage on all the franchisees, so their base costs are automatically way higher. They tried to expand into Ottawa and got crushed by PC Cyber, ShopRBC, etc. because those stores actually have good prices.
reply
milnoc @ 8th Oct 04:07PM:
Re: Blade servers
Thanks Guspaz. They're a bit far from where I am, so I'll get the missing spare drives from them later in the week. I already got the minimum two missing drives from MB this afternoon, and I now have enough equipment to start assembling everything together.
BTW, I'm filming the assembly and software installation so I can share the (painful) experience with all of you. :)
reply
Guspaz @ 8th Oct 04:14PM:
Re: Blade servers
Looks like about a 25 minute walk from you (basing your location on the H2Z postal code prefix), or 15 minutes by bus?
If you want them to shift stock to Montreal, I'd suggest you call them right now to reserve them and find out when they can get them here.
reply
olebiker @ 8th Oct 04:15PM:
Re: Blade servers
said by milnoc :
Thanks Guspaz. They're a bit far from where I am, so I'll get the missing spare drives from them later in the week. I already got the minimum two missing drives from MB this afternoon, and I now have enough equipment to start assembling everything together.
BTW, I'm filming the assembly and software installation so I can share the (painful) experience with all of you. :)
That is good I like it when people invent new curse words :)
reply
milnoc @ 8th Oct 04:16PM:
Re: Blade servers
Sorry! No audio track! Just background music! :D
reply
olebiker @ 8th Oct 04:17PM:
Re: Blade servers
said by milnoc :
Sorry! No audio track! Just background music! :D
Likely a wise move.
reply
milnoc @ 8th Oct 05:11PM:
Re: Blade servers
Aside from the DHCP not working, the Ubuntu installation is proceeding normally.
reply
milnoc @ 8th Oct 05:46PM:
Re: Blade servers
Reason why DHCP didn't work: network card not recognized. Intel has provided a driver and set-up instructions.
reply
milnoc @ 8th Oct 07:19PM:
Re: Blade servers
Third attempt at reinstalling the OS. This is actually not unusual. :)
I can't install the build environment because I don't have access to the Internet. But I need the build environment to create the network drivers and gain access to the Internet. Catch-22.
However, you can install the required software when you install the operating system. So I'm installing again. I'm also including LVM in the partitioning. Apparently, I need that for the software RAID to work properly.
I'm managing! :)
reply
nigrunze @ 8th Oct 07:29PM:
Re: Blade servers
said by milnoc :
Sorry! No audio track! Just background music! :D
You're going to use the CRTC muzak, right?
reply
milnoc @ 8th Oct 07:36PM:
Re: Blade servers
LOL!
Should I? :)
reply
mlerner @ 8th Oct 08:06PM:
Re: Blade servers
said by milnoc :
LOL!
Should I? :)
Lots to choose from. »music.muzak.com/music/programs/ Please DO NOT order country or classic!
Unfortunately Chum licenses Muzak in Canada. :huh: »www.chum.ca/products/muzakmusic.html
reply
milnoc @ 8th Oct 09:44PM:
Re: Blade servers
That's okay. I already have a piece chosen for the background music.
»incompetech.com/m/c/royalty-free···er+party
reply
milnoc @ 9th Oct 01:41PM:
Re: Blade servers
Quick update. Can't get the NICs working. The Intel e1000e drivers don't work under Ubuntu. I'm now using a D-Link DUB-E100 USB Ethernet adapter to gain network connectivity and load the full desktop installation where it'll be much easier to find a solution.
At worst, I can still install a couple of PCIE adapters. Anyone know which ones work with Ubuntu and don't cost a fortune? Or I can just stick with the cheap D-Link adapters and wait until a proper update is made available.
No news yet from TigerDirect on what might have happened to those four missing hard drives. Very weird.
reply
TSI Gabe @ 9th Oct 02:09PM:
Re: Blade servers
e1000e is supported under the linux kernel. You may simply have to custom compile your kernel, that's all.
reply
milnoc @ 9th Oct 02:16PM:
Re: Blade servers
I did follow the installation instructions included with the driver, but no soap. I couldn't even ping any machine on the local network. I'll try it again once the desktop and all the security updates have been installed. I work better with a GUI environment.
reply
milnoc @ 9th Oct 05:12PM:
Re: Blade servers
Found it!
The network cable was connected in the wrong jack on the computer. :D
No, it didn't take me this long to figure out. I installed the GUI and Webmin so that I could visually see what was happening.
I also had a very long lunch. :)
Next up, the RAID! The second server will be the guinea pig for the "pull-out-a-drive-and-see-what-happens" test.
reply
milnoc @ 9th Oct 05:40PM:
Re: Blade servers
Just realized how I made that mistake with the network jacks. eth0 is on NIC #2, and eth1 is on NIC #1.
They're reversed!
reply
milnoc @ 9th Oct 09:12PM:
Re: Blade servers
Still working on the RAID. It's more complicated than I thought. Apparently, I need to create three RAID arrays for the system to work. A 1 GB RAID-1 for the /boot, A 80 GB RAID-0 for the swap, and a 2.7 TB RAID-5 for the root filesystem, all created in that order.
I went for a RAID-5 instead of a RAID-10 so that I could gain an extra 1 TB of HD space. That will give me my minimum three months of television programming storage. I'll still format the second server as a RAID-10 for testing purposes. Whichever RAID works best for the file repository, I'll use it on a permanent basis.
There was even an option for a RAID-6, but I'll keep that for the 24 disc archive system. :D
reply
milnoc @ 9th Oct 09:54PM:
Re: Blade servers
A crazy idea.
There's a standard USB port inside the server mounted directly on the motherboard, with enough clearance to plug in one of those tiny USB drives in it.
What if I loaded /boot on that?
Any opinions?
reply
anon @ 9th Oct 10:16PM:
Re: Blade servers
Probably not a bad idea. Just have to make sure that the connector doesn't come loose under vibrations.
There are also those inst boot linux modules which is essentially a USB key with internal USB connectors. They might be a better fit if you run into height issues.
»clipmarks.com/clipmark/1622F55F-···7C01D3E/
reply
milnoc @ 9th Oct 10:49PM:
Re: Blade servers
I was hoping to use something like this. The only problem with leaving it inside the computer is that if it fails, I have to pull out the server to replace it. The other option is to leave it installed outside of the computer, which shouldn't be a problem since the rack will be is locked up.
My main issue with the RAID is what happens if the first drive goes bad just as I need to reboot the computer? I've installed GRUB on all four drives, but I still have to test out the configuration (i.e. pull out the first drive) to see if it actually works.
I'll try the USB approach on the second server tomorrow. That's the whole purpose of working all weekend long: get the RAID working properly before the datacenter install next week.
Sony USB drive |
olebiker @ 9th Oct 11:07PM:
Re: Blade servers
The good flash drives seem to stand up quite well. The transcend brand came to me highly recommended and has worked in a rough environment for quite a while for me.
reply
milnoc @ 10th Oct 12:47PM:
Re: Blade servers
The RAID works, even after pulling out the first drive and rebooting the system.
My only problem now is that I don't know how to initiate an automatic remount and rebuild of the RAID without having to reboot the system. Searching for the answer on-line is no help since everyone who tries to explain how to use mdadm seems to be on some kind of mind-altering drugs!
Then again, that pretty much describes the entire Linux community. :D
But hey! The Linux software RAID works perfectly! And that's what I needed to know more than anything else.
reply
milnoc @ 10th Oct 03:57PM:
Re: Blade servers
The USB /boot option is out. The system crashed during reboot when it mounted the USB drive in the middle of the boot and corrupted itself. I was an interesting idea though.
My previous attempts to crash the RAID however have been pretty much futile. As long as GRUB is installed on at least the first two drives, the system will reboot normally. However, I would still prefer to fix the RAID without the need of a reboot. The least downtime, the better.
One more test to do: pull out and reset the first drive to null (no GRUB, no partitions, no nothing), reload it, reboot, and fix the RAID. If the system can still reboot normally after all of that, then my RAID design is finalized. I can relax and concentrate on the software installation and configuration.
I appreciate how the hardware has not become an issue with this setup (aside from the mismatched network ports :) ). Everything is pretty much off-the-shelf, and it all works as expected.
Is anyone actually reading all of these progress reports? :)
reply
zacron @ 10th Oct 04:23PM:
Re: Blade servers
said by milnoc :
Is anyone actually reading all of these progress reports? :)
I am :huh: --- I'm having alot of fun reading and what not... it is the long weekend.
reply
oxymoron69 @ 10th Oct 04:28PM:
Re: Blade servers
I've been keeping up with your reports....
I'm really interested in the mechanics of setting up a tv channel and you fill in a lot of the blanks that I had in my understanding of how things actually work.
You have a pretty good (entertaining) writing style, I'm sure there's more than a few of us keeping tabs on your progress.
reply
mlerner @ 10th Oct 04:33PM:
Re: Blade servers
said by milnoc :
Is anyone actually reading all of these progress reports? :)
Purely out of curiosity. That and I love seeing the dreaded IBM tales.
reply
milnoc @ 10th Oct 07:37PM:
Re: Blade servers
Okay, the RAID is now problematic.
I pulled out the first drive, put it in a toaster, and wiped out the data so that the drive would look brand new.
Not only did I fail to recover the system, but I screwed up the /boot RAID-1 partition along with it. The system failed to reboot.
I'll figure it out. It should be something very simple. Luckily, rebuilding the test system only takes half an hour at the most.
reply
milnoc @ 10th Oct 10:09PM:
Re: Blade servers
The simple answer? One simple change in the BIOS.
Main -> IDE configuration -> Configure SATA as [AHCI]
That makes the drives fully hot swappable. As soon as you insert a new drive, the OS picks it up immediately. It's then a matter of rebuilding the original partitions on the drive, assigning them to their corresponding virtual RAID devices, and resetting the MBR on the drive in case it becomes the boot drive.
If the LED lights on the drive don't indicate which drive went down, running "scsiadd -s" will identify exactly which drive you need to swap.
reply
olebiker @ 10th Oct 11:50PM:
Re: Blade servers
said by milnoc :
Is anyone actually reading all of these progress reports? :)
Yep. I will be interested in seeing your finished product.
reply
milnoc @ 11th Oct 01:23AM:
Re: Blade servers
I'm still recording video material. I should have something posted by the end of the week.
The RAID configuration is locked in. I was able to extract, reset and rebuild just about any drive in the server without any downtime whatsoever. That BIOS tweak made all the difference. I now have a RAID-5 array of 2.9 terabytes in size.
All the software is loaded. Tomorrow, everything will be configured and ready for the installation of my custom software once it's completed. I'll also be loading Smoothwall Express on my old Eee PC 900 as a temporary solution until the Supermicro server arrives later in the week.
Almost there!
reply
milnoc @ 11th Oct 02:25PM:
Re: Blade servers
Smoothwall won't work on the Eee PC 900. I'll use an old Linksys wired router instead. Smoothwall WILL work on the Supermicro server. One commenter at Newegg (I think) has already confirmed this.
reply
olebiker @ 11th Oct 02:30PM:
Re: Blade servers
Are you keeping track of the new swear words invented yet :)
reply
milnoc @ 11th Oct 03:16PM:
Re: Blade servers
No. There aren't any. I see computers as a game, not as a pain. :)
And if there were any swear words, they'd all be in French Canadian! :D
reply
olebiker @ 11th Oct 04:51PM:
Re: Blade servers
said by milnoc :
No. There aren't any. I see computers as a game, not as a pain. :)
And if there were any swear words, they'd all be in French Canadian! :D
I am with you there I rarely get angry at hardware, sometimes software can get me pissed. I grew up in Quebec and although I really don't have much french the little I do is likely not fit for mixed company :)
reply
Vinch @ 12th Oct 09:05AM:
Re: Blade servers
Did you mention a raid-0 for swap? Your system can't possibly happy to be using swap and see it disappear if a disk fails? You might want to check that out...
reply
milnoc @ 12th Oct 09:57AM:
Re: Blade servers
I made both the /boot and the swap RAID-1. I changed my mind about using RAID-0 for the swap. As you pointed out, what if the memory contents of a critical program are corrupted by a failing drive?
The final drive partition and RAID configuration for the four-drive array are as follows.
- 1 GB RAID-1 for the /boot (overkill, but I like round numbers :) )
- 20 GB RAID-1 for the swap (suitable for 6 GB memory, or for 12 GB memory if I add a second processor)
- 975 GB RAID-5 for the "/" (root), giving me a total of 2,925 GB storage space in that one partition.
Note: in this case, 1 GB = 1 billion bytes, and not 1.074 billion bytes.
Total time required for a rebuild: just over four hours.
There's still about 3 to 4 GB of free space left at the end of each drive, but I'm leaving that blank in case I need to replace a drive, and the new drive is slightly SMALLER than the original drive. If I had used up all of the available space, a smaller drive would have killed any possibility of rebuilding the RAID-5 array.
RAID-6 and RAID-10 weren't practical on a system with such a small number of drives. Plus it would have reduced my total storage capacity by almost a terabyte. However, the future archive system which will consist of between 16 and 24 hard drives will definitely require a RAID-6 at the very least.
reply
InvalidError @ 12th Oct 11:59AM:
Re: Blade servers
said by milnoc :
I made both the /boot and the swap RAID-1. I changed my mind about using RAID-0 for the swap. As you pointed out, what if the memory contents of a critical program are corrupted by a failing drive?
If a drive starts spitting out corrupt information but not outright failing, three scenarios can happen:
1) RAID1 in performance mode, parallel writes with round-robin (or other distribution algorithm) read commands: the corrupt data will simply pass through
2.1) RAID1 in error-checking mode, parallel read and writes: the controller compares data from both drives, re-issues the reads in case of mismatch and flags whichever drive returns inconsistent data or exhibits excessive latency as failing
2.2) If the error occurred on write though, both drives would consistently return mismatching data and RAID1 would have to either pick one or issue a read error
reply
milnoc @ 12th Oct 12:02PM:
Re: Blade servers
What does mdadm use? That's the software RAID solution I'm currently using.
reply
InvalidError @ 12th Oct 01:27PM:
Re: Blade servers
said by milnoc :
What does mdadm use? That's the software RAID solution I'm currently using.
Most RAID solutions that I have read about support either mode. RAID5 usually has optional read parity checking to cut latency and save power so you may want to check up on that as well.
Modern transaction-based interfaces and HDDs have nearly end-to-end ECC built into them so soft read/write error rates should be extremely low, in all likelihood good enough that you should not need to lose sleep over performance-mode RAID1 for swap.
If you are that concerned about data corruption, you may want to reconsider RAID5 for primary storage since it too would run into trouble if it encountered a persistent parity mismatch. Only RAID6 provides some protection against drive-level data corruption.
reply
milnoc @ 12th Oct 03:58PM:
Re: Blade servers
Considering I have two matching servers instead of one for redundancy purposes, I may as well leave it as it is. I've yet to encounter any problems on any of my other computers equipped with either Hitachi or WD drives.
As for Maxtor, Seagate and Fujitsu, I have had problems with them. Then again, a couple of those failures were user generated. Plugged a failing Maxtor drive into a modified power connector so that I can test it out. Ended up feeding the 5 volts pathway with 12 volts by mistake. The burning smell was the dead giveaway. :)
I'm almost done with the set-up. I just need to install the OpenVPN configuration files, and configure the temporary router.
And if anyone wants to know why I'm doubling up on all the hardware, just look at what happened with T-Mobile's Sidekick in the US. »Sidekick Users Get Outage AND Total Data Loss
:D
reply
nigrunze @ 12th Oct 04:08PM:
Re: Blade servers
Don't worry about it milnoc. Chances are that you'll be fine as long as long as nobody logs into the guest account. :p »www.engadget.com/2009/10/12/snow···rorizin/
reply
milnoc @ 12th Oct 04:22PM:
Re: Blade servers
BTW, because of the problems and delays in obtaining the computer equipment, I had to reschedule my launch date for November 13, 2009.
November 13 is a Friday. :D
reply
nigrunze @ 12th Oct 04:29PM:
Re: Blade servers
said by milnoc :
I had to reschedule my launch date for November 13, 2009.
That gives you 15 days to put on Ubuntu 9.10. :D
reply
milnoc @ 12th Oct 04:54PM:
Re: Blade servers
LOL!
Actually, I'm waiting for Ubuntu 10.4 LTS. And no, I won't go the upgrade path. I tried that on the server at work. From 8.04 LTS to 9.04 in three complicated steps. What a horrible mess that was...
Yes, I had backups. :)
reply
milnoc @ 12th Oct 11:31PM:
Re: Blade servers
That's it!
OpenVPN is installed on both servers and are functioning properly. All that's left is a backup of both systems, and the servers will be ready to be installed at the datacenter. The BitTorrent configuration will be done on site where I can properly monitor it in case the Internet connection gets saturated by a rogue transfer.
The old Linksys BEFSX41 router is a real dog. It's a good thing it'll soon be replaced by a router that will actually work.
reply
Guspaz @ 13th Oct 10:30AM:
Re: Blade servers
Odd, I took a server from 7.04 to 8.04 LTS (a VPS no less).
Other than some issues with Xen in 7.10, I managed to do it successfully without any permanent issues.
Upgrades between LTS releases (IE, 8.04 -> 10.04) are supposed to have a single upgrade step; you'll be able to go directly between them.
In fact, I find that Ubuntu upgrades on servers works much better than on desktops!
reply
milnoc @ 13th Oct 10:42AM:
Re: Blade servers
I've installed the ubuntu-desktop package on my office server. That server is the guinea pig for software development. Chances are I broke something pretty severely with my frequent recompiles of the VLC Media Player and its dependencies.
I didn't try anything unusual with 9.04, and I don't plan to. So the office server will be a good test bed for an upgrade from 9.04 to 10.04 LTS.
Yes, I've installed ubuntu-desktop on the two new servers as well. What can I say? I'm a GUI whore! :D
reply
Guspaz @ 13th Oct 10:50AM:
Re: Blade servers
You really don't need ubuntu-desktop to do UI stuff on a server unless you really want the full desktop UI tool set.
For example, installing uTorrent on a server can be achieved nicely with just the basic x.org stuff, WINE, and nx.
Installing a desktop should be possible by just installing gnome and a few other essentials. Otherwise you're getting crap like OpenOffice pulled in on your server!
reply
milnoc @ 13th Oct 12:42PM:
Re: Blade servers
Are you suggesting I should use VIM instead?
I'd rather have my toenails pulled out with rusty pliers soaked in lemon juice! :D
I prefer having a GUI handy, especially for monitoring the broadcast streams once the channel is up and running. Command shells simply can't give me the info that I need in an efficient manner.
As for OpenOffice being loaded on the servers, that's not a problem. It isn't doing anything, and it can be useful at times.
Even with the GUI, I have to admit I haven't used a command shell this much since my OS/2 days! :)
reply
Guspaz @ 13th Oct 01:06PM:
Re: Blade servers
Huh? Vim? No... Where did that come from?
I'm implying that installing the ubuntu-desktop on a server is hugely wasteful and pulls in hundreds and hundreds of megabytes of bloat that are useless to servers. And will probably cause conflicts by the fact that ubuntu-desktop, being designed for a desktop, will load in all sorts of auto-perform stuff like the network manager and such things that you don't want on a server.
reply
milnoc @ 13th Oct 08:25PM:
Re: Blade servers
It isn't a problem. Linux is so efficient, the CPU registers zero most of the time. And I can always remove the components that aren't needed.
Also, this is for a TV channel. You can't monitor a broadcast on a text based screen -- unless you insist on watching an ASCII Art presentation. :D
THE SERVERS ARE ON THEIR WAY TO THE DATACENTER!
However, I still have the hard drives. Didn't want to risk them. I'll bring those myself, one server at a time.
Yes, all the drives are properly labeled. :)
reply
nigrunze @ 13th Oct 08:59PM:
Re: Blade servers
said by milnoc :
Also, this is for a TV channel. You can't monitor a broadcast on a text based screen -- unless you insist on watching an ASCII Art presentation. :D
lol. I'd like to see that.
What Guspaz was suggesting is, rather than installing the Ubuntu-Desktop package, you can just install GNOME and a few other packages that you think you might need. The Ubuntu-Desktop package pulls in LOTS of dependencies. Do you really need Oo.o, or Pidgin (Empathy in 9.10), or Evolution? Those are just a few examples.
And then there's still Xfce (or even more alternatives) as an alternate desktop environment.
Good luck with your launch and keep us updated.
reply
milnoc @ 14th Oct 08:19AM:
Re: Blade servers
said by nigrunze :
lol. I'd like to see that.
Considering this is Linux, there just might be something like that out there. :)
Yes, I'll be removing some of the software over time. It's just much easier for me to load the entire Ubuntu desktop and remove what I don't need, than it is to load a minimalist desktop and finding out later the software you installed manually is still missing a few optional dependencies required to make it work the way you want.
Also, I have 2.6 TB of storage space in each of those two beasts. So what does it matter if the desktop uses 5 GB more space than it should? :)
reply
milnoc @ 14th Oct 03:31PM:
Re: Blade servers
Both servers have been installed at the datacenter, and one of them is running. The second one will be activated in a couple of hours from now, and I'll then be able to verify if I have an Internet connection or not (nothing's plugged in right now).
Almost there!
reply
milnoc @ 14th Oct 05:05PM:
Re: Blade servers
Installed the drives for the second server. Both servers are operating normally.
No internet connection yet, but that was expected. It'll only be plugged in tomorrow. I went ahead and configured the router with all of the appropriate information and left the cable plugged into the router.
The worst that could happen at this stage is that the Linksys router takes down the entire city of Montreal. :D
reply
milnoc @ 15th Oct 11:10AM:
Re: Blade servers
Internet's working, but it's flaky as hell. The most likely culprit is the Linksys router. Its firmware hasn't been updated in years.
Actually, even with the latest firmware installed, chances are the router will still perform very poorly. It's a good thing it'll be replaced next week with a proper firewall appliance. Still, I need that thing to work properly for the next few days, so it's a trip to the datacenter today!
reply
milnoc @ 15th Oct 06:25PM:
Re: Blade servers
Went to the datacenter and upgraded the router's firmware. It helped a bit, but not by much. I then plugged both the ISP cable and the router's WAN port into the unmanaged switch. Suddenly, perfection.
Cogent's gear and the router's 10 Mbps WAN port are incompatible. Luckily, this won't be a problem next week. The only hiccup is that the network cable will be too short to reach the firewall appliance. I'll have to use an extension, or connect both the Cogent and the WAN cables in the switch.
Fixed a couple of minor issues and started uninstalling applications that weren't really needed in the desktop such as OpenOffice, the games, and the Network Manager.
Now I have to find an upgraded version of lm-sensors. Unfortunately, the server is too new for the current version, so I can't monitor the fans or the system temperatures. The only other way I may be able to monitor the systems is through the serial ports. A USB serial adapter should do the job without having to shut down the system and crack open the case.
That's pretty much it for now. The systems are operational, and I can start working on live prototypes of my broadcast system.
reply
milnoc @ 15th Oct 07:23PM:
Re: Blade servers
Does anyone here know where I can find the instructions required to configure postfix to send all system generated emails via an external email server? I really don't need to run my own, especially since the broadcast servers don't even have proper domain names attached to them.
reply
Vinch @ 15th Oct 08:44PM:
Re: Blade servers
Try this:
»embraceubuntu.com/2005/09/07/set···postfix/
reply
milnoc @ 15th Oct 11:25PM:
Re: Blade servers
Thanks! I'll try it out tomorrow.
That reminds me. I need to create a permanent OpenVPN connection between the office and the datacenter so that I don't have to connect manually every time I want to do something. With its ridiculous RSA key support, it should be safe enough.
Famous last words. :)
reply
milnoc @ 16th Oct 08:18PM:
Re: Blade servers
Ok. Forget the office wide OpenVPN coverage. OpenVPN needs to reside on a router for office wide coverage to work properly, but my office router is currently loaded with Tomato/MLPPP which doesn't include the OpenVPN support.
No problem. I already have OpenVPN installed on all of the essential office computers, and it works fine. I'll deal with the office-wide support only when it becomes a necessity.
Still haven't figured out how Postfix works. To be honest, it was much easier for me to program my own SMTP emailer application years ago! :)
I might leave the Linksys router at the datacenter for now, and use the tiny Supermicro server as a prototype for the automated broadcast client appliance. Surprisingly, I'm not the only company who uses an off-the-shelf consumer router in a datacenter. In fact, some racks have tower cases jammed in them for crying out loud! :)
Next up: the video. I'll work on it on Monday. The only shot that's left to film is the installation at the datacenter. And yes, I do have permission to film my installation as long as I'm escorted, and no other client's equipment is visible in the shots.
I insisted on it. ;)
reply
evil_gusgus @ 18th Oct 04:24PM:
Re: Blade servers
said by milnoc :
Ok. Forget the office wide OpenVPN coverage. OpenVPN needs to reside on a router for office wide coverage to work properly, but my office router is currently loaded with Tomato/MLPPP which doesn't include the OpenVPN support.
No problem. I already have OpenVPN installed on all of the essential office computers, and it works fine. I'll deal with the office-wide support only when it becomes a necessity.
Still haven't figured out how Postfix works. To be honest, it was much easier for me to program my own SMTP emailer application years ago! :)
I might leave the Linksys router at the datacenter for now, and use the tiny Supermicro server as a prototype for the automated broadcast client appliance. Surprisingly, I'm not the only company who uses an off-the-shelf consumer router in a datacenter. In fact, some racks have tower cases jammed in them for crying out loud! :)
Next up: the video. I'll work on it on Monday. The only shot that's left to film is the installation at the datacenter. And yes, I do have permission to film my installation as long as I'm escorted, and no other client's equipment is visible in the shots.
I insisted on it. ;)
We dont want no terrorists sabotaging the data center lmao.
reply
milnoc @ 18th Oct 06:48PM:
Re: Blade servers
Actually, I almost sabotaged my own service! I just realized the Linksys router will NEVER survive all of the BitTorrent traffic that's going to be pumped through it. So I got an Asus WL-520GU router and loaded Tomato 1.25 on it.
reply
twizlar @ 18th Oct 11:20PM:
Re: Blade servers
That thing will die as well. You need enterprise gear to push that many PPS.
--
Broadline Networks Inc.
reply
milnoc @ 19th Oct 01:00AM:
Re: Blade servers
It's a temporary measure to get me by for now. Besides, my Internet connection is only 10 Mbps. By the time I actually need 100 Mbps and a more effective router, I'll already have the Smoothwall appliance installed. Until then, I want to see if I can turn that little Supermicro server into an effective broadcast client while I have it in the office.
reply
milnoc @ 19th Oct 07:32AM:
Re: Blade servers
Interesting situation. Cogent is emailing ME about packet loss! But since most of my communications are BitTorrent-based where packet loss can be rampant (and seamlessly handled by the BitTorrent protocol), it may actually be a non-issue.
reply
milnoc @ 20th Oct 03:38PM:
Re: Blade servers
Here's the video. Didn't get any shots of the datcenter. Didn't feel like wasting a hour of my time to get a measly ten seconds of shots. :)
»www.youtube.com/watch?v=i7ACwp9mxng
reply
Guspaz @ 20th Oct 04:49PM:
Re: Blade servers
Handled? Yes. Seamlessly? Depends on your definition of seamless.
Packetloss will kill a TCP connection regardless of the application-level protocol. If your swarms are heavily reliant on your initial seeding, your throughput going down the tubes isn't ideal. It may be less of an issue on tracker updates, though. I'd also urge you to leave DHT/PEX/etc enabled (don't mark your torrents as private), ensuring that clients will be able to access the content even if your tracker is down/overloaded/slow/inaccessible.
reply
milnoc @ 20th Oct 06:06PM:
Re: Blade servers
I'm planning to use my office server as a backup, and even my home computer if necessary.
That's assuming I don't get dinged by UBB. :p
BTW Guspaz, would you have a recommendation for a Linux BitTorrent app that can work with RSS feeds, both generating and reading them? I'm currently using Azureus Vuze, but I'm not very happy with its performance. It's a bit unstable and a resource hog.
It's too bad uTorrent isn't available on the Linux platform. That might have solved all of my problems. And no, I'm not interested in trying Windows. The last thing I need is the WGA halting all system operations.
reply
Guspaz @ 20th Oct 09:46PM:
Re: Blade servers
When I ran Linux (Ubuntu 9.04, most recently) as my primary OS, I did use uTorrent under WINE. It works quite well, and both WINE and uTorrent have had bug fixes to work with eachother. DSL_Ricer also runs uTorrent on his home server (which runs Debian).
I guess Deluge is a popular one these days?
reply
milnoc @ 21st Oct 02:06AM:
Re: Blade servers
I'll check both of them out. Thanks!
I especially like the Web GUI approach.
reply
milnoc @ 21st Oct 03:13PM:
Re: Blade servers
Received the Supermicro 5015A-H today!
I'll be performing media playback tests on it this afternoon just to get a feel of its capabilities. If it can handle 720p material, it'll be perfect as an intermediate high definition broadcast appliance. It's doubtful it'll be able to handle compressed 1080i material, but you never know.
As for the sound card, I'll simply use a Behringer USB audio interface. I've already confirmed it'll work properly under Ubuntu. No need to try and jam a sound card in that tiny box.
I doubt anyone's reading this right now what with the two CRTC decisions published today. :D
reply
Guspaz @ 21st Oct 04:12PM:
Re: Blade servers
So, MPEG-2 decoding? You should get hardware acceleration for that out of the Intel chipset. If you use the right codec, 1080i may be possible. Besides, MPEG-2 is easy to decode as long as you don't futz with it (apply filters). Of course, it couldn't transcode it or 720p, all it could do is decode, not sure how useful that is.
reply
milnoc @ 21st Oct 05:27PM:
Re: Blade servers
I'll be using a combination of codecs including MPEG-2, although I would prefer to avoid using MPEG-2 altogether if I can. It simply doesn't perform as well as today's more efficient codecs such as MPEG-4.
It may even be possible to stream standard definition material in its raw format to the cable headends, consuming no more than 5 Mbps of bandwidth.
High definition broadcasts however will be a real pain. I'll probably need at least 17 Mbps of bandwidth to make that work.
Anyway, I'll try out a few things with that little box during the next couple of days. The worst that could happen is that it simply can't do the job. In that case, it'll become the firewall appliance at the datacenter, which is why I bought it in the first place.
So many toys, so little time. :D
reply
Guspaz @ 21st Oct 06:54PM:
Re: Blade servers
Anything broadcast related will be MPEG-2. Raw uncompressed SD video will be ~250Mbps.
reply
milnoc @ 21st Oct 07:07PM:
Re: Blade servers
Sorry. Poor choice of words. I meant in its original compressed format (DivX, XviD, H.264, MOV...) with no transcoding whatsoever.
Imagine it as a file copy.
reply
milnoc @ 21st Oct 07:54PM:
Re: Blade servers
Quick update. The Supermicro can handle 720p, but it can't handle 1080i (actually, 1080p from a deinterlaced source). No surprise there, that's the limitations of the Atom processor.
Now I have to figure out how to eliminate the tearing in the image. This is a very old Linux bug that STILL hasn't been resolved yet.
I can always fall back on (gasp!) Windows if necessary, but I would prefer to avoid that option. That would mean a 50% increase in the unit cost of the appliance along with all the licensing and WGA headaches that come with it.
reply
Guspaz @ 22nd Oct 02:10AM:
Re: Blade servers
Yikes! Linux media playback is in a bad state.
Performance will also be much better under Windows. XP, specifically; I noticed a slowdown in media playback going from XP to Vista. I also noticed, when I was dual-booting, much better performance on Windows than Linux.
You're also far more likely to get any hardware acceleration working under Windows. I assume that you don't have an ION based Atom, but you might still eke some DXVA use out of it.
With CoreAVC for h.264, you'll get most 1080p content playing back (it's multithreaded).
reply
milnoc @ 22nd Oct 06:45AM:
Re: Blade servers
I can try XP and 7 on it. There might also be a couple of options left under Linux such as ffmpeg. But yes, video playback under Linux is still a major problem, one that has stopped me from adopting Linux full-scale some years ago.
Luckily, this is not a complete show-stopper. VLC is available on both platforms along with its powerful CLI capabilities, and I've created multi-platform apps in the past. But I'm worried that Windows' paranoid WGA and license protection schemes will suddenly shut down the national broadcast.
Windows 2000 maybe? :)
reply
Guspaz @ 22nd Oct 11:57AM:
Re: Blade servers
I'm not sure 2000 is the best idea (isn't it EoL'd?). WinXP is probably still the most optimal platform for media playback.
I'd question the quality of VLC on any platform, as I think many people would ;)
ffmpeg uses libavcodec, which ultimately virtually everything on Linux and Windows are based. mplayer, xine, VLC, ffdshow, Media Player Classic, they all use libavcodec.
WinXP's speed advantage over Vista probably has something to do with the extra overhead and DRM in the media chain. As for Linux, well, GCC is a *much* worse compiler than Visual C++ or ICC, so that should account for at least pat of the speed. A very inefficient display chain is another reason.
reply
milnoc @ 22nd Oct 12:18PM:
Re: Blade servers
I'll try Windows 7 RC1, and turn off the UAC and the Aero. Unfortunately, Win XP will soon suffer the same fate as Win 2000. Plus, Win XP driver availability might start to become a problem once all the computer manufacturers start to move over completely to 7.
Of course, I was kidding about using Win 2000. The driver issues with that OS have become a real nightmare in the last few years. But you'd be surprised how many businesses still rely on it for mission critical applications. In fact, some places are still using NT4 for crying out loud! The BS I had to deal with at my last job to ensure that applications were compatible under both NT and 2000 drove me totally batty!
reply
milnoc @ 22nd Oct 01:36PM:
Re: Blade servers
It looks like Windows 7 is the winner. The videos play flawlessly under the VLC media player except for the 1920x1080 material. The average CPU usage while playing my 1280x720 material (i.e. my Canon/Sanyo camcorder shoot-out review) is about 17% with a peak CPU usage of 22%.
Note that this computer has a 1.6 GHz dual-core Intel 330 Atom processor in it. There are four CPU usage graphs in the task manager! :)
Since this will be a broadcast client machine, it'll require none of the server or business functionality commonly seen in enterprise machines. A cable company can save themselves a few bucks by purchasing a Windows Home license instead of the more expensive corporate or ultimate licenses.
reply
mlerner @ 22nd Oct 01:41PM:
Re: Blade servers
I had a feeling Windows 7 would be the best choice. It's getting rave reviews in performance from netbooks to high powered workstations. If you do need server functionality though, 2008 R2 is equally as good in performance.
reply
milnoc @ 22nd Oct 01:49PM:
Re: Blade servers
I've been using 7 RC1 on my Sony Vegas editing station for a while now. No problems to report except for the annoying preview scanner I believe it's called. It keeps wanting to scan the huge MTS files, and jams up the desktop in the process. However, that won't be a problem at the client sites.
I'll stick with Linux on the datacenter servers. It can easily handle both the streaming and BitTorrent server duties, the RAID support is built-in, the nags can be turned off, and I save a huge wad of cash on the licensing fees.
But for the client sites, it'll definitely be Windows.
reply
valunthar @ 22nd Oct 06:52PM:
Re: Blade servers
Might be a bit of a moot point since you've already chose to use Windows 7 as your platform. But you can get CoreAVC running under linux.
Info:
»ubuntuforums.org/showthread.php?t=839295
»en.wikipedia.org/wiki/CoreAVC
reply
milnoc @ 22nd Oct 11:01PM:
Re: Blade servers
Running CoreAVC for 1080p material isn't the problem. It's the tearing that's the problem. If the computer's A/V outputs are connected directly into the cable headend, the tearing will be visible at people's homes.
For streaming and transcoding from a Linux to a Windows computer however, I've yet to see any image defects. I still have a series of tests to complete before I can confirm this, but it's looking good.
The other advantage of using a Windows machine for video playback is that if I need a specialized graphics cards for broadcast purposes, chances are there will be plenty of drivers for the Windows platform, and absolutely nothing for the Linux platform.
BTW, the stereo sound card will consist of a Behringer U-Control UCA202 USB audio interface. Thirty-seven bucks, plug-and-play even on a server, works on both Windows and Linux, ultra-low latency, no perceivable background noise.
I wish I knew this before I paid $150 for a Creative sound card that doesn't work under Linux. :)
reply
anon @ 22nd Oct 11:50PM:
Re: Blade servers
>Running CoreAVC for 1080p material isn't the problem. It's the tearing that's the problem.
I don't know what software you are using to play your video.
In Media Player Classic Homecinema, you can set "Options/playback/Output/DirectShow Video" to the new EVR (enhanced video renderer) mode available in Vista/Windows 7 which is supposed to minimize(/eliminate???) the tearing.
»mpc-hc.sourceforge.net/AntiTearing.html
reply
Guspaz @ 22nd Oct 11:40PM:
Re: Blade servers
If disk speed on your editing station is such a big problem, you may want to consider an SSD, or
maybe even one of the new $900 Fusion-IO boards.
I'd be very cautious about mixing very heavy random access things like BitTorrent with real time things like streaming or broadcast servers. BitTorrent can hit the disks really really hard if you have a lot of content to stream and it can't all fit in the disk cache.
reply
milnoc @ 23rd Oct 12:14AM:
Re: Blade servers
said by An_Onymous :
I don't know what software you are using to play your video.
In Media Player Classic Homecinema, you can set "Options/playback/Output/DirectShow Video" to the new EVR (enhanced video renderer) mode available in Vista/Windows 7 which is supposed to minimize(/eliminate???) the tearing.
We're talking about tearing under Linux. Windows doesn't have that problem as long as you use at the very least the Overlay option.
In fact, both platforms have shown no noticeable difference in playback performance during the last few years. Good is still good, and bad is still bad. The software doesn't seem to make much of a difference as long as you use the proper renderer.
reply
milnoc @ 23rd Oct 12:25AM:
Re: Blade servers
said by Guspaz :
If disk speed on your editing station is such a big problem, you may want to consider an SSD, or
maybe even one of the new $900 Fusion-IO boards.
Disc speed isn't the problem. I have a similarly configured machine at home, but it's running XP instead of 7. It has no speed issues whatsoever. This particular problem only started happening when I began dumping M2T files on the Windows 7 machine.
As for disc wear, streaming and BitTorrent, my machine at home has been using the same 500 GB Hitachi Deskstar drive for the last two years, and it's still going strong despite frequent video playback, constant BitTorrent activity, and a few other things. It doesn't seem to mind the constant activity as long as the heads aren't constantly traveling all over the platters in the process.
Famous last words. KE-RASH! :D
Yes, I have off-site backups. :)
reply
Guspaz @ 23rd Oct 10:44AM:
Re: Blade servers
It's not a matter of failure, it's a matter of excessive random accesses. It's *very* different to be doing torrent seeding on hundreds of gigabytes of content for business use rather than a handful of gigabytes for home use. People downloading things are going to cause seeking all over the place.
Your swarm sizes will probably be small enough to not have an issue, though.
reply
milnoc @ 23rd Oct 10:52AM:
Re: Blade servers
That, and all the latest files will be located in the same area of the discs. That's why I'm not worried.
When the system gets big enough that excessive random accesses will become a problem, that's when I'll implement a few neat tricks to reduce the thrashing.
reply
milnoc @ 23rd Oct 06:29PM:
Re: Blade servers
Does anyone know a site that would have a decent multicast HOWTO?
I'm able to stream with a reasonable amount of reliability at up to 5 Mbps, including H.264 video and AC3 audio. There might already be devices on the market that can properly interpret a stream generated by the VLC Media Player, which would greatly simplify the system design. Buy a box, enter the proper IP address, connect to the headend, and you're done.
reply
Guspaz @ 24th Oct 04:29PM:
Re: Blade servers
Your upstream provider would need to support multicast, I don't think Cogent does. If they do, they surely charge extra.
When you're talking about a stream to the BDU, that's 24/7, you'll probably just have to suck it up and pay the transit fee. I'm not sure what kind of a commit you made to Cogent, but I assume it's under 100mbit, so you're paying $7-10 (depending on the length of the commit). I'll assume 1-year at $9 per megabit, so $45 per month per BDU. That seems pretty cheap, although since you're not charging anything to the BDU...
I assume that all your stream-to-endusers stuff is all BitTorrent and they don't get a live stream? As much as I'd love to get a live stream, I can see how that would cost a small fortune to distribute.
On the other hand, there has been research into live P2P streaming. Tribler has experimented with live BitTorrent-based streaming (SwarmPlayer), and Bram Cohen (creator of BitTorrent) announced about a month ago that he has decided to turn his attention to live streaming. His goal is less than 5 seconds of latency on live streams.
I was actually somewhat concerned when I read that you intend to allow content creators to control whether their content appears over the net. This will prevent you from having a live stream that carries your broadcast stream over the net.
reply
milnoc @ 24th Oct 06:49PM:
Re: Blade servers
So the cost of multistreaming is still based on how much total bandwidth you can provide to everyone. It's not like an OTA transmitter that can reach an unlimited number of receivers. I was afraid of that.
Anyway, the streaming component will be split in two. A high quality stream reserved for the BDUs, and a low quality stream for the general public if I can afford it.
If I can't afford it, or if the public wants a higher quality product than the live stream, they'll have to use BitTorrent. It's currently the most economic and most efficient content delivery system in existence.
Giving content providers a choice of distribution technology is necessary if I want them to submit material for broadcast consideration. A lot of the established production companies still don't understand the Internet and its potential, preferring to rely on antiquated and outdated business models to conduct their affairs. They will simply not want to see their content delivered unrestricted to every piece of technology out there because they don't understand it.
My contract with Cogent is per month. I can drop them at any time if they turn out to be stinkers. But up to now, I have no complaints. I've yet to experience any problems with the connection although Cogent was a bit worried when their routine ping tests against my $55 router indicated two dropped packets per 1,000 pings. :)
What's really stupid with Cogent is that they're CHEAPER, FASTER and MORE STABLE than my two TSI office lines! $100 per month for 10/10 service compared with $126 per month for 10/1 service!
Unfortunately, Cogent's downtown fibre deployment doesn't reach my office building. If it did, I would have chosen them in a heartbeat.
reply
Guspaz @ 24th Oct 11:05PM:
Re: Blade servers
said by milnoc :
So the cost of multistreaming is still based on how much total bandwidth you can provide to everyone. It's not like an OTA transmitter that can reach an unlimited number of receivers. I was afraid of that.
Sadly, yes. And even if your ISP supports multicast, the ISP of people receiving the stream need to support it too.
Anyway, the streaming component will be split in two. A high quality stream reserved for the BDUs, and a low quality stream for the general public if I can afford it.
You'd bankrupt yourself trying to do a live stream at any quality. If you were to assign a quarter megabit per user, which is VHS quality and probably about as low as you can afford to go (typical SD content for torrents is over one megabit, and that's VBR while streams are CBR or ABR), and a hundred people try to tune in, you're suddenly using 25mbit/s.
If I can't afford it, or if the public wants a higher quality product than the live stream, they'll have to use BitTorrent. It's currently the most economic and most efficient content delivery system in existence.
I won't disagree with BitTorrent's efficiency, although it doesn't yet handle live streams well. SwarmPlayer does, but it's not there yet. I have confidence that Bram Cohen will do for streaming what he did for distribution. Hopefully some time next year, you'll be able to P2P stream live a high quality feed to everyone. You won't need 5mbit/s for SD streaming to actual endusers. Even one megabit per second is probably good enough for a high quality stream, maybe two.
Giving content providers a choice of distribution technology is necessary if I want them to submit material for broadcast consideration. A lot of the established production companies still don't understand the Internet and its potential, preferring to rely on antiquated and outdated business models to conduct their affairs. They will simply not want to see their content delivered unrestricted to every piece of technology out there because they don't understand it.
That becomes a logistical issue. It means that your online stream is no longer the same as your broadcast stream. You then need to be turning on and off the live stream all the time and that gets annoying. You may want to simply include live streams as "broadcasts". They're not typically stored or random-access. They go out when it's broadcast and then they're over. If a content producer doesn't want his stuff put up as a torrent, even if they don't understand it, telling them "Your content will be broadcast live through various mediums, but will not be downloadable on the internet" is probably easy enough to understand.
My contract with Cogent is per month. I can drop them at any time if they turn out to be stinkers. But up to now, I have no complaints. I've yet to experience any problems with the connection although Cogent was a bit worried when their routine ping tests against my $55 router indicated two dropped packets per 1,000 pings. :)
What's really stupid with Cogent is that they're CHEAPER, FASTER and MORE STABLE than my two TSI office lines! $100 per month for 10/10 service compared with $126 per month for 10/1 service!
You're not actually supposed to say what you're paying Cogent, I believe you would have signed an NDA with them. I only know of Cogent's pricing because of this chart that was published:
»telephonyonline.com/mag/telecom_···ws_down/
Does Hurricaine Electric have service in your datacenter? They have a $2/mbit price. The problem is, I think, is that you have to do BGP routing.
Unfortunately, Cogent's downtown fibre deployment doesn't reach my office building. If it did, I would have chosen them in a heartbeat.
They don't seem to really have any lit office buildings based on their website.
reply
twizlar @ 25th Oct 12:44AM:
Re: Blade servers
You can get non-bgp(their ips) from HE for $2.00/mbit @ 1000mbit commit or they'll charge you about $600 for 100mbit.
--
Broadline Networks Inc.
reply
milnoc @ 25th Oct 12:44AM:
Re: Blade servers
Cogent's pricing is not under any NDA. In fact, they market themselves as being the $7 per megabit company. *
* Based on a three year contract. :)
Cogent is already available in many downtown Montreal buildings. However, any list you may find on-line is most likely already out of date. For my particular building, I know it isn't available. But it is available across the street at Place Ville-Marie. Unfortunately, the rent there is through the roof, and the building's management has a strict policy against filming inside the building, which is a bit of a problem for a TV station. :)
Hurricane isn't available at Canix, but these other companies are. See any you like?
•3Men@Work
•Allstream
•Altima
•B2B2C
•Bell Canada
•CANARIE
•Choopa
•Cogeco
•Cogent Communications
•Colbanet
•Cooptel
•Cyberlogic
•Digital Days
•Dotality
•Fibernetics
•Fibrenoire
•Fibroptix
•Globalive
•Gogax
•Hibernia Atlantic
•Hydro One Telecom
•IBM
•Internap
•IPSosoft
•Iron Mountain
•Level (3)
•Limelight Networks
•MCI
•Metro Optic
•MTO Telecom
•Net Axis
•Netelligent
•NLayer Communications
•Nost
•Openface Internet
•Orange
•Pixel Web
•Recall
•RISQ
•Rogers
•SRDP
•SunGard
•T-Systems
•TATA Communications
•Teltech 2000
•Telus
•Terago Networks
•Tinet International Network
•Uniserve
•Vermontel
•Videotron
•VIF Internet
•XIT Telecom
•Zerofail
Some time next year, I'll need to hook up with an alternate provider in order to provide full redundancy to the entire system.
As for live streaming via BitTorrent, I'm aware it isn't ready yet, so I won't use that part of the technology for now. I'll be happy enough that people can at least download the programming and watch it at their leisure. And the data I've accumulated during my sharing of the CRTC throttling hearings will help in estimating how much bandwidth the system will need to properly distribute the channel's programming.
I'll keep in mind your idea on how to sell the broadcast, streaming and BitTorrent components. Combining the broadcast and live streaming into a single entity would solve a lot of legal headaches.
BTW, one of the computer programs I'm writing is the broadcast automation core. The core will be fed a programming schedule specific to the target distribution platform (live, stream, torrent), and will include instructions as to whether a particular show should be processed or not. The core will then directly control the VLC Media Player through the Telnet interface, feeding it the source files and any special playback instructions if required.
The VLC Media Player, despite its complexity, is a pretty remarkable piece of software. On-screen playback, redirected to a file, or streamed on the Internet, it pretty much does it all.
Still gotta figure out how to create those RSS feeds for the torrents though. I know it's an XML file, but there's no way in H-E-Double-Hockey-Sticks I'm gonna code all of that crap manually! Maybe there's a Wordpress plug-in that can do the job...
reply
Guspaz @ 25th Oct 03:43AM:
Re: Blade servers
Creating an RSS feed is pretty trivial. The format is pretty simple, so you just need a loop spitting the things out.
You just need to spit out the header part:
Then you have a loop, one per piece of content, that outputs in this format:
And then you spit out the footer:
So really it's just a "fill in the blanks".
reply
milnoc @ 25th Oct 10:10AM:
Re: Blade servers
What ever happened to the simplicity and readability of comma delimited?
(Yes, I'm revealing my age here. :) )
reply
Guspaz @ 25th Oct 03:18PM:
Re: Blade servers
CSV is still useful for moving data around ;)
BTW, saw this story today:
World's first BitTorrent powered live-streamed concert
Things are progressing. I figure in 6-12 months, this stuff'll be reliable enough for production use.
reply
milnoc @ 2nd Nov 02:05PM:
Re: Blade servers
An update.
First, don't assume that just because the manual indicates a part is "optional" that it wasn't already included with your purchase. Not only was the DVD drive included with the servers, but so was the ASMB4-iKVM remote administration module!
Second, server "Tuesday" may have a problem. I've discovered this while installing Ubuntu 9.10 yesterday, and the incident forced me to head to the datacenter to perform some emergency troubleshooting.
What happens is that if I try to reboot the computer within Ubuntu using the remote console in the ASMB4-iKVM's Web interface, the BIOS screen won't even show up, and the computer remains frozen.
Even worse, if you log out from the ASMB4's Web interface while the computer is frozen, you won't be able to log back into the interface because it's lost access to the user profiles. However, this appears to be a problem only with older ASMB4 firmware versions such as 1.10 and 1.5. I've loaded version 2.03 in the ASMB4 this morning, and I haven't had any problems with it. But I still have to perform a couple of tests to be certain it's working properly.
With the server completely inaccessible, I had to head back to the datacenter and unplug the server in order to reset it properly.
I tried the same series of reboots on server "Monday", and had no problems whatsoever. The only difference between the two systems are the hard drives. Hitachi Deskstars on Monday, and WD Caviar Blacks on Tuesday.
The ASMB4 module is very powerful. You can still access the Web interface even when the computer is turned off, and turn it back on again. Just make sure you have the latest firmware loaded on the ASMB4 otherwise you risk being locked out of your system.
I'll switch the power supplies later in the day and try the tests again. Tuesday's event logs have been spitting out alerts with the 5 volt line slipping out of spec. I don't know if this is an issue with the power supply, or with the WD drives.
If anyone from Asus is reading this, any suggestions? I don't believe it's the BIOS battery since my settings remained intact through all of these incidents. And there were no error LEDs lit up inside the server during the incident or during the reset.
reply
Guspaz @ 2nd Nov 03:28PM:
Re: Blade servers
You probably want to be using a network-enabled PDU so that you can do hard-reboots remotely.
Something like this:
»www.apc.com/products/resource/in···u=AP7931
or this:
»www.apc.com/products/resource/in···u=AP7900
reply
milnoc @ 2nd Nov 04:08PM:
Re: Blade servers
I already have that. It's called "The Datacenter Staff." :)
Switched the power supplies. Nothing to report at this time. I'll keep an eye on the event logs. One thing's for sure, the WD drives do consume a LOT more power on the 5V line! 420ma each for the Hitachi drives, 680 ma for the WDs!
Once I install the spare power modules, that might eliminate the problem altogether. So we'll play it by ear for now. Maybe Asus will release a firmware upgrade that will stagger the spin-up of the hard drives.
reply
milnoc @ 2nd Nov 08:48PM:
Re: Blade servers
Nothing in the logs yet. Both servers are clean. However, I can't perform a reboot test at this time. I'm in the middle of a project with the VLC media player and the Videolan Manager (vlma) Web interface.
reply
Guspaz @ 3rd Nov 12:18PM:
Re: Blade servers
I wouldn't exactly call slightly more than one watt "a LOT more power"... 420ma and 680ma at 5V are both extremely small amounts of power (2.1W and 3.4W respectively).
reply
milnoc @ 3rd Nov 12:32PM:
Re: Blade servers
What about the power spike when the system starts up? Four hard drives all engaging their spindle motors at the same time? That might be just enough to cause a power drop across the 5V line. Most upper-scale RAID systems spin up the drives one at a time, avoiding the power spikes.
Along with the power supplies, I also swapped the ASMB4-iKVM module in the affected server. I had previously ordered and already received a module before discovering the servers already included them. :p I figured it wouldn't hurt to try it out.
Still nothing in the logs.
reply
Guspaz @ 3rd Nov 04:01PM:
Re: Blade servers
My bad, hard disks are on the 12V rails, not 5V. So that power requirement would be 5W and 8.2W respectively.
Some brief googling shows that ~2.25A isn't atypical for spin-up power requirements. That'd be ~27W.
A good power supply should be able to handle a dozen such drives spinning up at the same time, but not everybody has good power supplies...
I believe that Linux' RAID subsystem can/does stagger startup, though; DSL_Ricer's setup does that, causing enormous delay when he tries to spin up drives
reply
milnoc @ 3rd Nov 04:22PM:
Re: Blade servers
In my case, powering up or rebooting the servers automatically spins up every single drive at the same time. I might simply have a weak power supply in my set-up. If it becomes a problem, I'll get Asus to replace the PSU under warranty.
reply
milnoc @ 3rd Nov 07:51PM:
Re: Blade servers
It's the power supply.
I just tried to reboot both servers. Tuesday rebooted without a hitch, but Monday jammed up. There's no doubt about the source of the problem.
Now comes the next test: getting proper support from Asus. :)
reply
milnoc @ 3rd Nov 09:55PM:
Re: Blade servers
Okay. Asus support can be a bit flaky at times, but it's not that bad once you find someone who knows their business.
The North American parts office is closed right now, so I'll have to call them tomorrow morning to have them ship me a replacement power supply. I'll also ask them if they can find me the second power supply for each server. They've been on back-order for a while with no ship date in sight.
Both servers are running now. I just cant reboot the affected server without someone being there to babysit it. Luckily, it's loaded with Linux and not Windows :p, and the datacenter is only a few minutes away by foot. So it's no problem for me to do the work myself especially if I wait until the end of the day before performing the reboot.
And if anyone's wondering if things would have been better with IBM, not necessarily. I've once had a brand new IBM computer with a defective riser card constantly corrupt my OS/2 installation on a regular basis for three weeks until we realized what was the cause. And then there was the expensive SCSI RAID server with the bad motherboard. Not very reassuring when you've just unpacked the thing.
A power supply is easily replaceable, so I'm not worried. And I didn't get any "it's your fault/you don't know what you're doing" type of arguments whatsoever from Asus.
reply
milnoc @ 4th Nov 12:35PM:
Re: Blade servers
Regrettably, Asus has just earned an automatic FAIL in the mission critical category.
They won't send me a replacement power supply. Instead, they want me to send the defective power supply to Markham, Ontario to be repaired. And because there are still no redundant power supplies available anywhere in North America (everything is back-ordered), that means the affected server will be DOWN for at least two weeks.
The purpose of redundant components is to keep the systems running even when something fails. But when both the replacement and redundant parts aren't available in a timely manner (within 24 hours), the mission critical aspect of the entire product is completely shot.
Automatic FAIL.
reply
An_Onymous @ 4th Nov 12:49PM:
Re: Blade servers
Looks like you as the system integrator would have to take up the slack by stocking extra systems and/or parts. :(
There is fundamentally something funny about the hard drive spin up that cause a dip in the +5V rail. Most HDD has motor and the voice coil on the +12V rail while the control circuits are on the +5V. I played a bit with a 300W ATX supply on a old dual PIII system with six 3.5" PATA HDD as ZFS 5+1 and a 2.5" HDD for booting FreeNAS. I have not seen power rails issues.
The redundant power supply would likely help a bit in your case assuming it is a load sharing power supply.
reply
milnoc @ 4th Nov 01:13PM:
Re: Blade servers
Even though it was a possibility, the drives were never an issue. This was proven when I swapped the power supplies, and the problem started cropping up on the other server. The original "defective" server with the "good" power supply was finally working properly.
I've just updated my YouTube video with the latest info.
reply
mike12806 @ 4th Nov 02:50PM:
Re: Blade servers
Can you repost your YouTube links?
reply
milnoc @ 4th Nov 02:59PM:
Re: Blade servers
»www.youtube.com/watch?v=i7ACwp9mxng
reply
Vinch @ 4th Nov 03:17PM:
Re: Blade servers
You mentionned that one server sometimes jams up when you reboot it. The disks should not be stopping and should not have to spin back up again. I'm thinking you have another issue with the machine. BTW, a test you could do is to pull a disk or two out and see if it consistently starts up propoerly.
reply
milnoc @ 4th Nov 03:25PM:
Re: Blade servers
Read the last few posts. I've already confirmed it was the power supply all along.
reply
Guspaz @ 4th Nov 03:44PM:
Re: Blade servers
I'm sure you intend to continue discussing with Asus why this is not an acceptable response? Insist that you want them to cross-ship a replacement part (you'll have to give them your credit card info as collateral, this is standard practice).
This is clearly not an acceptable situation, and you need to continue pursuing it.
BTW: If the redundant PSUs are identical to the primary PSUs (and they should be), and they're still manufacturing/selling new servers, then the whole "backordered" thing is bullshit. They're putting the PSUs into new machines, so they can damned well send you replacements. They exist, they're available, they just don't want to give them to you.
reply
milnoc @ 4th Nov 04:03PM:
Re: Blade servers
Unfortunately, the two PSUs are NOT exactly the same. The first PSU has a slight "hump" on top that prevents it from fitting inside the second PSU's slot. However, it's possible the second PSU will fit perfectly into the first slot since the hump serves no other purpose than to segregate the first PSU from the second slot.
But since all the second PSUs are back-ordered, there's none available for either PSU slot. And the first PSUs are most likely all allocated to the retail server production run.
I have no idea why they built two different sized PSUs when internally they're exactly the same. I might do a video just to point that out. One of my friends also pointed out that due to the recession, it's possible Asus may have cut back production on certain items and simply ran out of power supplies.
Don't worry, I'll keep nagging them until they get this issue resolved to my satisfaction. To their credit however, they weren't too difficult to reach by phone, and everyone spoke proper English with only one person having a recognizable but not too thick non-English accent.
reply
milnoc @ 6th Nov 01:00AM:
Re: Blade servers
Asus found a power supply. I'll send them the paperwork in the morning.
Great timing! I just launched a test broadcast yesterday which after a few tweaks should look like a proper cable television broadcast by the end of next week.
Things are moving VERY fast these days!
reply
milnoc @ 9th Nov 03:14PM:
Re: Blade servers
Arg. Received the replacement power supply. It didn't resolve the problem. I might have been misled by the origins of the problem, which I now suspect may simply be my dual OpenVPN connection used in conjunction with the ASMB4 remote module.
I'll still keep the power supply they've sent me and cancel the RMA. Until those two redundant PSUs are shipped, this PSU will be the only backup I have readily available.
This problem is EXTREMELY annoying!
reply
milnoc @ 9th Nov 07:39PM:
Re: Blade servers
Arg again. It appears that's exactly the problem. If you try to reboot a Linux computer using the ASMB4 interface via an OpenVPN connection to that same computer, the computer will freeze up completely after the shutdown and will NOT reboot no matter how many power cycles or resets you issue either through the interface or in person.
If however you use the ASMB4 interface to reboot the computer from the local network (i.e. not on OpenVPN), or if you reboot the computer from an SSH session via an OpenVPN connection, the computer will reboot normally.
It's ONLY when you use the ASMB4 interface and an OpenVPN connection TOGETHER that the computer will freeze up to the point you have to PHYSICALLY unplug the computer in order to reset it.
I was misled into believing the problem was with the power supply because of the connection sequence of the OpenVPN connections. The last connection established was for server Tuesday, the one that always jammed up. that connection became the dominant one of the two. But when I swapped the power supplies, the startup sequence caused server Monday to be the last one connected and become the dominant one. So when I tried to reboot Server Monday...
Jammed solid.
How the heck can a combination of separate and unrelated physical and virtual devices successfully deadlock a computer? This is really weird!
reply
oxymoron69 @ 17th Nov 05:16PM:
Re: Blade servers
"The WMP encoder is working on 11089, but not on 11084. These are two different servers that are SUPPOSE to be identically configured, so there must be a (stupid) problem with the first server. I also managed to break the WMP on one of my Windows 7 RC computers, so the testing of the stream MIGHT be hazardous to your WMP installation."
Both streams were working and showing video here using WMP in windows 7 ultimate until the video switched from you building servers to democracy now, at that point 11084 crapped out to just audio.
The other stream at 11089 scaled down the video when the file changed and continued on just fine.
BTW, great video quality on that self shot server building video.
edit:
I posted here because it's more topical than the other thread.
reply
milnoc @ 17th Nov 05:47PM:
Re: Blade servers
I may have found the problem. I believe I need to reset the VLC Media Player's configuration whenever I start it. If the --sout-keep option is left active by mistake, it'll prevent the transition from one video to another from happening within WMP.
Once DN is over, I'll reboot the servers (gotta perform a clean-up). The streams will restart at the next video.
The video won't break up anymore. I cut the frame rate back to 15. Enjoy the slide show! :)
reply
milnoc @ 17th Nov 07:05PM:
Re: Blade servers
Weird. If you boost the resolution past 240x180, it goes crazy on you. I may have to find another solution, such as Firefox's video tag. I'd rather use that knowing it should work on all platforms.
Both streams are working now.
Server Monday: »38.104.152.86:11084
Server Tuesday: »38.104.152.86:11089
Requires Windows Media Player.
reply
Guspaz @ 18th Nov 09:01AM:
Re: Blade servers
Low-bitrate low-res video looks bad ;)
reply
milnoc @ 18th Nov 01:18PM:
Re: Blade servers
Check out port 11084. I just made a few changes. I'll port them over to port 11089 later this afternoon. So whenever one feed isn't working, try the other one.
I also found out why I was having so many problems yesterday. Whenever you launch multiple instances of the VLC Media Player each with its own configuration parameters, you must assign each instance its own configuration file otherwise the player will use the default configuration file and screw everything up.
As expected, it WAS one stupid little detail that was the root cause of all my problems.
reply
Guspaz @ 18th Nov 02:34PM:
Re: Blade servers
I'm at work, and so I won't be able to check it out for a while.
What's your target stream format eventually? This morning, it was 240x180 (roughly one third of SD) at 310kbit (for both audio and video, I believe); not terribly watchable. Well, audio quality wasn't that bad, although I'm not sure why you were using MP3; virtually anything (WMA, AAC, Vorbis, etc) would be more efficient.
reply
milnoc @ 18th Nov 02:56PM:
Re: Blade servers
For now, I'm limited by the capabilities of both VLC and WMP. And VLC is a bitch to figure out! A ton of options, but no comprehensive instruction manual!
Just like Linux! :D
I still have to check out the video tag option in Firefox. If VLC supports it, I'll set up an encoding stream compatible with it, and set up a dummy Web page where we can test it out.
reply
Guspaz @ 18th Nov 03:15PM:
Re: Blade servers
I believe HTML5's VIDEO tag is targeted at progressive downloads rather than live streaming.
There is also the big codec snafu when it comes to the VIDEO tag. None of the browser manufacturers could agree on what codecs to support, so the W3C just threw up their arms and left it undefined.
In other words, browsers may have mutually exclusive support for codecs!
VLC should support arbitrary resolutions and bitrates. Codecs might be restricted by various things, but bitrates and resolutions should be independent of that.
reply
milnoc @ 18th Nov 07:37PM:
Re: Blade servers
Starting at 8 PM:
- Democracy Now for November 18, 2009 (launched at 8 PM on 11084)
- Bleeding Ears: Paxton Bachman Band
- Hi-Fi Expo 2008
- Short: Make Mine Freedom (launched at 8 PM on 11089)
- Movie: Suddenly
- Movie: Dishonored Lady
Both streams at ports 11084 and 11089 should be available all night long. Tomorrow however, I'll be working on server "Tuesday" (port 11089). So the stream will be interrupted on that channel.
Next in the system development: automating all of this crap! :)
EDIT 1: 11089 will be restarted later. I just discovered that even though VLC can play Ogg files, apparently it can't stream them. Very annoying.
EDIT 2: No movies tonight. The Internet Archives crashed. I can't download alternate versions.
reply
Guspaz @ 19th Nov 03:38AM:
Re: Blade servers
Looks like it's at 320x240 (quarter VGA, my bad, the previous res was much lower than one third SD/VGA) at 783kbit for the stream.
That bitrate *should* be almost sufficient for SD, but it looks like the current program was recorded on a cellphone/digicam, so the source is too low quality to judge.
You may be better off trying to use a real streaming suite rather than VLC... Something like Windows Media Services, QuickTime Streaming Server, Helix Server, etc.
All three support HTTP, WMS and Helix both support Windows Media codecs, Helix and QTSS both support generic MPEG-4 and h.264 and other codecs.
EDIT: Now it's on the audio convention, which is much better quality source. It looks like the video is "pulsing" every second or less (on a fixed interval). It's almost like it's sending out keyframes crazy frequently, which would waste a ton of the limited bandwidth. It'd make sense; it should look much better than this consider the bitrate.
EDIT2: That bit where you apologize for the room being so long because you like the soundtrack... The soundtrack sounds REALLY similar to the soundtrack of Star Trek II (composed by James Horner). At times, I almost thought it WAS a medly of the soundtrack, but it wasn't.
reply
milnoc @ 19th Nov 06:42AM:
Re: Blade servers
said by Guspaz :
EDIT2: That bit where you apologize for the room being so long because you like the soundtrack... The soundtrack sounds REALLY similar to the soundtrack of Star Trek II (composed by James Horner). At times, I almost thought it WAS a medly of the soundtrack, but it wasn't.
Translation: "I watched your channel." :)
Time to start inserting a few ads! :D
I might not use WMP at all; it's simply too restrictive. I'm searching for a "universal" low-res, low bandwidth streaming technology that's not only supported by multiple platforms at the receiving end, the server components must also work under Linux. Windows Server is not only an expensive and needlessly complicated proposition, I'm really worried its licensing and copy protection technology may accidentally shut down the broadcast.
On the high end however, VLC is perfectly suited. The picture quality of the 2.5 Mbps h.264 stream is simply astounding, and the load on the servers is much lower than originally expected. The only item that needs to be fixed is the inconsistent transition from one video to another during the transcoding process. That's the only area where VLC is causing major headaches.
Thanks for the feedback! Once The Internet Archives is back up, I'll start adding a few movies to the schedule.
reply
milnoc @ 19th Nov 12:32PM:
Re: Blade servers
BTW, not only did you watch my channel, you were also watching one of my shows. A CANADIAN show!
You will be assimilated. :D
reply
Guspaz @ 19th Nov 01:21PM:
Re: Blade servers
I had already watched the program on YouTube ;)
reply
Oinktastic @ 19th Nov 02:38PM:
Re: Blade servers
Sorry, how and where can I watch these streams?
reply
milnoc @ 19th Nov 02:58PM:
Re: Blade servers
Server "Monday": »38.104.152.86:11084
Server "Tuesday": »38.104.152.86:11089
Requires Windows Media Player.
If one stream doesn't work, try the other one.
If BOTH streams don't work, I screwed up. Wait a few minutes while I'm in Panic mode. :)
reply
Oinktastic @ 19th Nov 03:26PM:
Re: Blade servers
I should just be able to "Open URL" and then past your server's IP address and port in there, right?
I haven't been able to access them yet with WMP or VLC. Not sure if I'm doing something wrong.
reply
milnoc @ 19th Nov 05:24PM:
Re: Blade servers
And the "http://" at the beginning as well.
Under VLC, it has to start with "mms://". Or is it "mmsh://"?
Tonight and tomorrow, the stream will be available only on port 11084. I'll be trying out a few things on 11089 today and tomorrow, so the stream will crash constantly on that one.
I'm hoping to reroute the input through an equivalent of the "display" interface, then pass the output through the transcoder. VLC has a lot of trouble transcoding and streaming certain file formats, which is a big problem for me since I'll be dealing with an assortment of very bizarre file formats. I won't have time to convert everything.
At least my videos work! :)
reply
Oinktastic @ 19th Nov 05:36PM:
Re: Blade servers
Awesome. Video quality is a bit...
but I'm not really sure what to expect for a 800Kbps stream. I'll check it out from time-to-time. Neato!
Thanks.
reply
milnoc @ 19th Nov 05:43PM:
Re: Blade servers
Don't trust the quality of the video currently playing on port 11084. You see that light on the ceiling? That's the only lighting in the room! You can imagine the amount of noise in the original recording. :)
reply
milnoc @ 19th Nov 06:17PM:
Re: Blade servers
Tonight's schedule starting at 6:30 on port 11084 (if everything works properly -- cross fingers) :)
Democracy Now
Girls Beware - (Ephemeral series)
Salon Son & Image 2009 (The Trade Show Show)
Make Mine Freedom (Ephemeral series)
Private Snafu - Booby Traps (pre-movie cartoon)
Movie - Suddenly, crime drama with Frank Sinatra
Movie - Night of the Living Dead by George Romero
And commercials. :)
As previously mentioned, port 11089 will be unavailable until further notice. All you'll see is the channel's test pattern, or an experimental broadcast.
reply
dandin1 @ 19th Nov 06:31PM:
Re: Blade servers
Nice! I can see!
For others: I used VLC, went to Media>Open Network Stream>pasted 38.104.152.86:11084 into the box, and selected MMS from the dropdown.
reply
Guspaz @ 19th Nov 07:22PM:
Re: Blade servers
They're coming to get you, Barbara!
reply
milnoc @ 19th Nov 07:46PM:
Re: Blade servers
What do you think? Should I put on "Boys Beware" tomorrow? :)
EDIT: Crap. The commercials aren't working.
EDIT 2: Hold on. I gotta fix that. I'll increase the bitrate.
EDIT 3: It's working again. At this rate, I'll probably tell people to simply use the VLC Media Player. The bitrate for the WMP feed is already at 1 Mbps! I can get ten times better results on the 2.5 Mbps H.264 feed!
I'll work on this on Tuesday. Not the day of the week, but the server. :)
reply
Guspaz @ 19th Nov 09:22PM:
Re: Blade servers
1Mbps should be enough bandwidth for SD, and you're blowing it on one quarter the resolution! I really think you need to rethink your general-public-distribution...
Why aren't you using 1Mbps h.264 SD video for the public stream too? WMP should play that just fine in Windows 7, and it'll play it just fine in other Windows that have an h.264 decoder installed. Or, they can use VLC.
You're burning bandwidth at this point :(
reply
milnoc @ 19th Nov 09:38PM:
Re: Blade servers
There's a reason I'm using the DIV3/mp3 codecs. It's the only combination I could find that was compatible with WMP. I've tried many different combinations without success until I found the right one documented in the VLC Wiki.
Don't worry about the bandwidth. Only three people are connected including me (I'm watching the movie).
Night of the Living Dead will be slightly delayed. The file wasn't transferred to the right server. It's still at the office.
On the plus side, I found the problem with the commercials. Wrong paths! :D
reply
Guspaz @ 19th Nov 09:51PM:
Re: Blade servers
said by milnoc :
There's a reason I'm using the DIV3/mp3 codecs. It's the only combination I could find that was compatible with WMP. I've tried many different combinations without success until I found the right one documented in the VLC Wiki.
Wait... MS-MPEG3... DivX 3... WTF are you using this for?!? No wonder it looks so bad for this bitrate!
Is compatibility with WMP really so important?
Don't worry about the bandwidth. Only three people are connected including me (I'm watching the movie).
I'm not worrying about bandwidth, I'm worrying about quality. It looks pretty bad, and I'm trying to watch it :P
Why not offer both? You did say that CPU usage wasn't very high. Why not have three streams:
1) Broadcast stream, 2.5Mbps h.264
2) High-quality public stream, 1 Mbps h.264
3) High-compatibility public stream, 1 Mbps h.264
I have trouble believing that WMP will play DivX 3, but not WMV 8 (codec name WMV2). I mean, WMV 8 is Microsoft's own codec :P
reply
milnoc @ 19th Nov 09:55PM:
Re: Blade servers
I've been having a lot of trouble with VLC and WMP interoperability lately. But that's okay. The purpose of these test runs is to iron out all the bugs and dump what doesn't work.
WMP support isn't absolutely essential, but it would be a nice feature to have for Joe and Jane Six-Pack. :)
Next movie in about fifteen to twenty minutes.
If you have VLC, check out port 11083.
reply
milnoc @ 19th Nov 10:04PM:
Re: Blade servers
Just restarted VLC. I switched over to WMV2 instead of DIV3. It's the first time I see it work.
EDIT 1: Just tried WMV3. Nothing's happening. Going back to WMV2.
reply
Oinktastic @ 19th Nov 10:13PM:
Re: Blade servers
I'm just getting the of-the-air screen right now. Is that what I'm supposed to be getting?
reply
Guspaz @ 19th Nov 10:14PM:
Re: Blade servers
I'm not getting anything; WMP won't play it anymore (»38.104.152.86:11084/)
reply
milnoc @ 19th Nov 10:17PM:
Re: Blade servers
Stand by. Something's screwy here.
reply
milnoc @ 19th Nov 10:20PM:
Re: Blade servers
Rebooting the server. Somehow, the codec support got severely corrupted. I couldn't even restart the original WMP configuration.
reply
milnoc @ 19th Nov 10:30PM:
Re: Blade servers
This might take a while. I really don't understand what's happening. It's like something broke.
reply
Guspaz @ 19th Nov 10:35PM:
Re: Blade servers
I'll note that the reason why Otakuthon and Otaku Anime never use VLC for screenings is that it is notoriously buggy and unstable. When we did try to use it, it could not be trusted to be left unattended and run through the playlist; something was bound to be broken or crash.
reply
milnoc @ 19th Nov 10:41PM:
Re: Blade servers
I see what you mean. However, it is capable of running for days with the h264/a52 codec. I've yet to lock myself out of that one.
I'll keep trying to fix it. It has to be something stupid.
reply
milnoc @ 19th Nov 11:04PM:
Re: Blade servers
It's working again. Somehow, it no longer likes to be run in multiple instances. The WMV version was the second in the list, and it kept crashing. I killed the high bandwidth version, leaving the WMV version the only instance running.
I'm restarting it near the end of the first movie. That gives you TWO MINUTES to catch the beginning of Living Dead.
I'll investigate this bug tomorrow. I'm going to bed.
reply
Guspaz @ 19th Nov 11:42PM:
Re: Blade servers
It looks much better, but still QVGA. Have you tried 480p at 1Mbps?
Randomly: Your source for this movie is this, right?
»www.archive.org/details/Night.Of···ad_1080p
That's a raw BluRay rip.
reply
milnoc @ 20th Nov 12:01AM:
Re: Blade servers
Actually, it's this one. »www.archive.org/details/night_of···ing_dead More specifically, the Mpeg4 512k file.
The video's resolution is exactly the same as the stream's. 320x240. :)
I'll try different things tomorrow on port 11089. I want to cut the bitrate well below 800 kbps. However, I might fix the multiple instance failure issue first, followed by the design of a smarter processing chain that will allow me to properly support multiple source overlays.
So much work left to do. But I'm definitely making progress! Two weeks ago, I was barely getting started. Now I have enough knowledge to start designing the support software that will directly control the VLC.
reply
Guspaz @ 20th Nov 12:15AM:
Re: Blade servers
Generally, you'll want as high quality a source as possible. Even if you're encoding at low-quality. This is because if you re-encode a low-quality source, you're spending bandwidth compressing the artifacts and such.
It's generational loss taken to the extreme, like a copy of a copy of a copy of a VHS.
reply
milnoc @ 20th Nov 12:21AM:
Re: Blade servers
That's why I love to use material from archive.org so much. If I can make that crap look good, everything else will be fine. :)
reply
milnoc @ 20th Nov 11:34AM:
Re: Blade servers
If all goes to plan, tonight's schedule will include a ten minute short called "Cliché Family in Televisionland", and the movie "Beast from Yucca Flats" starring Tor Johnson.
Be advised that both features are rated 14+.
reply
milnoc @ 20th Nov 03:47PM:
Re: Blade servers
Two more movies in the rotation: "My Man Godfrey" and "Renfrew of the Royal Mounted."
Any preference for tonight?
reply
AkFubar @ 20th Nov 05:56PM:
Re: Blade servers
+1 for Renfrew! Where can i pick up your broadcast milnoc?
reply
milnoc @ 20th Nov 06:12PM:
Re: Blade servers
Let's see if the sig works this time. The addresses are in there.
TONIGHT'S SCHEDULE STARTING AT 6:30 PM
• Democracy Now for Friday, November 20, 2009
• Dining Room Table Reviews - Fujitsu LifeBook U810
• Ephemeral - Boys Beware (definitely doesn't represent the channel's opinions :) )
• European Vignettes from Yet Another Travel Show
• My Messy Shorts - Cliche Family in Televisionland (rated 14+)
• Free Movie - Renfrew of the Mounted (look like a very decent print)
• PSA - Nuclear preparedness for farmers
• Free Movie - The Beast of Yucca Flats, featuring Tor Johnson (rated 14+)
Then it starts over in an infinite loop. Next week, I'll try to set up a timed schedule.
I might also add the more creepy pieces from my collection of ephemeral films to the rotation, more specifically the two sex education films "As Boys Grow" and "Molly Grows Up." But I must warn you, these films are really, REALLY creepy! :D
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
AkFubar @ 20th Nov 06:37PM:
Re: Blade servers
Hey so far this is pretty good! :)
reply
milnoc @ 20th Nov 06:41PM:
Re: Blade servers
I'll monitor the broadcast from my home. I also want to watch Renfrew. :)
reply
zacron @ 20th Nov 06:42PM:
Re: Blade servers
looking good milnoc!
reply
MaynardKrebs @ 20th Nov 06:42PM:
Re: Blade servers
said by milnoc :
WMP support isn't absolutely essential, but it would be a nice feature to have for Joe and Jane Six-Pack. :)
Just have a link to an installable PC/Mac VLC player on your website.
reply
MaynardKrebs @ 20th Nov 06:44PM:
Re: Blade servers
said by milnoc :
I'll monitor the broadcast from my home. I also want to watch Renfrew. :)
+1
How about 'Reefer Madness'? ;-)
reply
AkFubar @ 20th Nov 06:50PM:
Re: Blade servers
said by MaynardKrebs :said by milnoc :
I'll monitor the broadcast from my home. I also want to watch Renfrew. :)
+1
How about 'Reefer Madness'? ;-)
Now that's a creepy movie! :D
EDIT:
Nuclear preparedness for farmers sounds pretty good too.
--
If my online experience is enhanced, why are my speeds throttled??
reply
milnoc @ 20th Nov 07:09PM:
Re: Blade servers
said by MaynardKrebs :
How about 'Reefer Madness'? ;-)
Already showed it along with Sex Madness this week. I'll show it again next week.
I love ephemeral films! In their attempt to "educate" people, they became some of the most hilarious -- and sometimes most shocking -- material ever seen in pictures!
My next target: You Can't Do That on Television. I'd love to get my hands on that classic, hopefully with the infamous "Divorce" episode!
One more thing: I HAVE AN AUDIENCE! It might be an audience of only two or three, but it's a start! It'll be very difficult to fall below that!
Thank you all for your continued encouragement!
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
zacron @ 20th Nov 07:12PM:
Re: Blade servers
Can you post something better then "Domocracy Now" it really is an annoying programme. I seriously think that 55% of the content should be from CANADA. lol.
IMO that is.
So far so good.
Cheers,
Zacron
--
-----------------------------
OK EVERYONE REPEAT AFTER ME
Goosefrabah.
Ahhh, feel better now? ;)
reply
mlerner @ 20th Nov 07:12PM:
Re: Blade servers
I'll probably tune in and out during the weekend. The low picture quality kinda bugs me though, given I'm on dual 21" screens with DVI and a whole lot more res and clarity.
reply
mlerner @ 20th Nov 07:13PM:
Re: Blade servers
said by zacron :
Can you post something better then "Domocracy Now" it really is an annoying programme. I seriously think that 55% of the content should be from CANADA. lol.
Well see.. if more people from Canada would do home made shows maybe that can happen.. a custom produced news show. It doesn't take much to record a show these days.
reply
AkFubar @ 20th Nov 07:17PM:
Re: Blade servers
said by zacron :
Can you post something better then "Domocracy Now" it really is an annoying programme. I seriously think that 55% of the content should be from CANADA. lol.
IMO that is.
So far so good.
Cheers,
Zacron
LOL yeah.... even reruns of Tommy Hunter or Don Messer would be better than Democracy Now
--
If my online experience is enhanced, why are my speeds throttled??
reply
milnoc @ 20th Nov 07:31PM:
Re: Blade servers
said by zacron :
Can you post something better then "Domocracy Now" it really is an annoying programme. I seriously think that 55% of the content should be from CANADA. lol.
I tried to locate some Canadian CC/public domain material on archive.org, but couldn't find any. But with a bit of luck and plenty of contributions, I'll reach my Canadian content target of 90% within three years.
For now, you'll have to put up with my own TV shows. And Democracy Now. :D
Democracy Now will be scheduled for 5 PM on weekdays once I get more programming, and the automated broadcast system is in a reasonable and functional state. Right now, I'm still working out the bugs, which is the main reason behind these broadcast tests.
But I must admit, I'm impressed with how well it works up to now. All I have at the datacenter is a 10 Mbps connection and an Asus router loaded with Tomato. And it all works!
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
zacron @ 20th Nov 07:33PM:
Re: Blade servers
lol.
I just cannot handle democracy now, it's a horrible piece of American "treasure"... I'd be willing to come up with something like, "Canada Now" (oh wait, cbc has that in public domain right?)
Cheers,
Zacron
--
-----------------------------
OK EVERYONE REPEAT AFTER ME
Goosefrabah.
Ahhh, feel better now? ;)
reply
AkFubar @ 20th Nov 07:33PM:
Re: Blade servers
I'm getting a very good quality stream using WMP.
reply
milnoc @ 20th Nov 07:45PM:
Re: Blade servers
You know what's really unnerving about this film? It's the music! It's so cheery as if something really nice will happen! :D
reply
mike12806 @ 20th Nov 08:20PM:
Re: Blade servers
Not too bad quality wise....
I can get the stream (Full Screen View) to open in VLC under OS X, but I do still get these errors:
access_mms: invalid host
main: no suitable access module for `»'
access_mms: cannot connect to server
access_mms: cannot read data
I typed »mms://38.104.152.86:11089 into the Network box.
Why aren't you using a domain name?
reply
milnoc @ 20th Nov 08:40PM:
Re: Blade servers
Setting up domain name access is a bit complicated. A simple IP address is the easiest way for me to perform my tests and not have to worry about connection issues or learning too many things at the same time.
BTW, port 11089 is a test pattern. Port 11084 is the movie. :)
I'll have to find a better encoder module some day. I'll probably have to bite the bullet and set up a Windows box at the datacenter. All I hope is that Microsoft's encoding software won't require a server version of their operating system. I'm not interesting in paying huge licensing fees for essentially nothing.
Yes, I should go with something else, preferably open/free software. But if the encoder can't work under Linux or the decoder requires installing extra software (something most users will be too afraid or too lazy to do), I may not have much of a choice.
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
MaynardKrebs @ 20th Nov 08:45PM:
Re: Blade servers
Here's one from archive.org that all Long-Gun Registry foes would love....
Rifle Marksmanship with M1 Rifle, parts 1 & 2
reply
InvalidError @ 20th Nov 09:37PM:
Re: Blade servers
said by milnoc :
Setting up domain name access is a bit complicated.
Go to dyndns.org, register a user name, create a domain name, enter the IP address, done in three minutes! (Well, maybe ten if it's your first time ever there.)
Getting a 'real' domain is just a little bit more complicated since you need to deal with top level registrars which means having to fill in a bunch of extra contact/billing info.
reply
milnoc @ 20th Nov 10:34PM:
Re: Blade servers
Getting a domain name is easy; that's just a purchase order. Where things get complicated is adding the /30 domain support to my connection along with installing and configuring all the required hardware and software. Cogent most likely can provide most of what I need, but I just don't have the time or energy to deal with all that crap at this time. Just give me my freakin' IP address and let me get on with my tests!
Once I hire some minions, they'll take care of that stuff. :)
This weekend, I need to find a way to play back the videos seamlessly. When played back on a computer monitor, no problem. But when you add in transcoding, it has trouble. I don't get it.
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
Spazomatic @ 21st Nov 01:57AM:
Re: Blade servers
Milnoc,
just read through this entire thread and must say I am very impressed with your project. Good luck with it!
Setting up a DNS server is really not that hard, especially with Ubuntu. If you need some help (free of charge) let me know would be glad to assist as I live in the MTL region (st-henri actually).
As other posters have mention, Vlan can be a bitch. I tried doing a multicast stream from by computer to the PS3 (which will not play PAL dvds) and it never worked that well... You may need to bite the bullet and buy a windows server where you would probably get more support or at least documenation. Maybe you can find someone to donate a license... (sorry don't have any!). I image you a looking for sponsor, you never know Microsoft may be willing to help for the free publicity.
Having said that maybe you can have a look at this:
»www.icecast.org though it only supports mostly open source codecs. in any case the stream you are pumping out now is not half bad.
Tim
reply
milnoc @ 21st Nov 08:56AM:
Re: Blade servers
said by Spazomatic :
Maybe you can find someone to donate a license... (sorry don't have any!).
Well then what good are you to me? Begone peasant! :D
The image may be good, but the bandwidth is way too high. The feed is currently set at 1 Mbps using WMV2 (a.k.a. WMV 8), which was forced upon me by the horribly encoded movie "Suddenly". Its artifacts eliminated most of the compression.
If I do need to use Windows for the encoding of the publicly available stream (CPAC is successfully broadcasting a 480x360 stream at 322 kbps), I might still be able to use VLC as the base. One thing VLC CAN do is play a lot of different video sources, both files and live streams. But just try to feed WMP an XviD/mp3 file, and it's confused. However, I might still be able to use VLC to feed WMP a high bandwidth local stream it understands, which it can then re-encode for public distribution. This way, I'll only need one Windows box.
Stay tuned. I have a few interesting ideas to try out that might fix everything. I just need some time. VLC may be a complicated bastard, but it's still the software solution that gives me most of what I need right away for my channel. Once the revenue stream comes in, I'll assign a team to fix and upgrade VLC to supports all the tricks major network systems currently support.
Yes, I'm putting "bugs" in all four corners of the screen! Muahahahahahahaha!!! :D
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
Guspaz @ 21st Nov 10:32AM:
Re: Blade servers
WMP will play almost any file that VLC will if you've got the right filters/splitters installed. I suggest you install Haali's Splitter, and the latest version of ffdshow tryouts... if you're using Windows. Then, WMP should play almost anything.
reply
milnoc @ 21st Nov 11:00AM:
Re: Blade servers
said by Guspaz :
I suggest you install...
*BZZZT!* :)
Remember, Joe and Jane Six-Pack are VERY afraid to install ANY software on their computers!
Too bad they also have the most virus-riddled computers in the universe. :)
BTW, I have Zoom Player installed on my Windows computers, which automatically installed ffdshow, haali, vsfilter, and a few other things. As a consequence, WMP also supports most a/v file formats although it does give a warning when you try to play MKV files.
Just thought of something. Can VLC produce FLV streams? It might actually be the better solution. WMP is starting to get on my nerves.
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
Guspaz @ 22nd Nov 01:59AM:
Re: Blade servers
Why are Joe and Jane six-pack installing software on your servers? Those suggestions were for Windows Media Services to be able to decode content.
I highly suggest against using codec packs for any sort of a professional installation, and in general. You should know exactly what software is on your hardsware.
I'm not a fan of FLV streaming. It means that users will need to view video in their web browser. It means that instead of using the player of their choice, they have to use YOUR website. It means that playback will consume much more resources, and many lower end computers won't be able to view your content. It means that Linux and Mac users will get a very sub-optimal experience, since Flash is quite poor/slow on those platforms.
reply
milnoc @ 22nd Nov 11:17AM:
Re: Blade servers
Oh! It's for the server! My mistake! :)
No problem about avoiding codec packs at all costs. I've been burned by them before. The makers of Zoom Player were also very much aware of this fact when they upgraded their software a few years ago. Zoom Player will only install codecs from their original sources, and not from any "packs." As a result, I'll be using ZP to install all the required codecs.
I'll keep Flash as an option however. The public option will remain an easily manageable low resolution stream so that people are encouraged to get the proper channel from a cable company. I'd only go full resolution on-line if I can't get enough cable companies on board and I'm left with no other choice.
That scenario might not happen however. Giving my channel away for free has actually helped me sell the channel. I currently have five separate cable companies willing to help me out with this project as best as they can, with the possibility of a few more coming on board. This doesn't mean they'll actually carry my channel, but it does mean they're open to new ideas.
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might (will) crash.
Main feed: 38.104.152.86:11084 Auxiliary feed: 38.104.152.86:11089
reply
Guspaz @ 22nd Nov 05:54PM:
Re: Blade servers
Even if some cablecos sign up, many people won't have a cableco or satellite company that carries the channel (or maybe they don't have cable or satellite at all).
Why should those who choose not to get cable TV be unable to watch a decent quality feed? I don't fall into this category because I've got Videotron, but forcing your viewers to resort to outdated "old media" sources for your channel is disheartening.
reply
milnoc @ 22nd Nov 06:14PM:
Re: Blade servers
It might be disheartening, but that's the way it has to be.
My greatest potential audience reach is still with the cable television audience. And the cable companies won't appreciate me undercutting their business by offering the general public an on-line broadcast feed that's just as good as their cable broadcast feed.
Limiting the resolution of the on-line public feed is purely a business decision.
BTW, I've just refreshed the broadcast schedule and added nearly all of the material I've presented during the last week. Here's the list of files since I'm now too lazy to write everything out. :)
setup programming input "/repository/programming/Democracy Now/dn2009-1120.mp4"
setup programming input "/repository/programming/TCPub/FujitsuU810Final.avi"
setup programming input "/repository/programming/TCPub/HPP1505ReviewFinal.avi"
setup programming input "/repository/programming/TCPub/AsusEeePC900HQFinal.avi"
setup programming input "/repository/programming/TCPub/AsusEeePC900HAReviewHQFinal.avi"
setup programming input "/repository/programming/TCPub/AsusEeePC1000HEReviewHQFinal.avi"
setup programming input "/repository/programming/TCPub/CanonHF200SanyoVPCTH1ShootOutFinal.avi"
setup programming input "/repository/programming/TCPub/RodeVideomicCamcorderMicrophoneWindTests30kmhGustsFinal.avi"
setup programming input "/repository/programming/TCPub/AsusRS700-E6RS4UnboxingAndAssemblyFinal.avi"
setup programming input "/repository/programming/Ephemeral/Girls Beware.avi"
setup programming input "/repository/programming/Ephemeral/Boys Beware.avi"
setup programming input "/repository/programming/Ephemeral/Boy With a Knife.mpeg"
setup programming input "/repository/programming/Ephemeral/Reefer Madness.avi"
setup programming input "/repository/programming/Ephemeral/Sex Madness.avi"
setup programming input "/repository/programming/Shorts/Oddball/Cliche Family in Televisionland.mp4"
setup programming input "/repository/programming/Movies/Theatrical/My Man Godfrey.mp4"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart1Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart2Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart3Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart4Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart5Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart6Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart7Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart8Final.avi"
setup programming input "/repository/programming/TCPub/EuropeanVignette1Final.avi"
setup programming input "/repository/programming/TCPub/EuropeanVignette2Final.avi"
setup programming input "/repository/programming/TCPub/EuropeanVignette3Final.avi"
setup programming input "/repository/programming/Movies/Theatrical/Renfrew of the Royal Mounted.mp4"
setup programming input "/repository/programming/TCPub/SSI2009HQPart1Final.avi"
setup programming input "/repository/programming/TCPub/SSI2009HQPart2Final.avi"
setup programming input "/repository/programming/TCPub/SSI2009HQPart3Final.avi"
setup programming input "/repository/programming/TCPub/SSI2009BonusRoomsHQFinal.avi"
setup programming input "/repository/programming/Ephemeral/A is for Atom.mpeg"
setup programming input "/repository/programming/Ephemeral/Make Mine Freedom.mpeg"
setup programming input "/repository/programming/Shorts/Private Snafu/Booby Traps.mp4"
setup programming input "/repository/programming/Movies/Theatrical/Suddenly.mp4"
setup programming input "/repository/programming/Movies/Theatrical/Night of the Living Dead.mp4"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand1Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand2Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand3Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand4Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand5Final.avi"
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might crash.
Main feed: »38.104.152.86:11084 Auxiliary feed: »38.104.152.86:11089
reply
Guspaz @ 22nd Nov 08:20PM:
Re: Blade servers
Then hurry up and start offering BDUs an HD feed so that you can offer us online users an SD feed :P
reply
milnoc @ 22nd Nov 08:22PM:
Re: Blade servers
Oh man, the bandwidth costs of HD are going to KILL ME!!!!! :D
reply
Guspaz @ 22nd Nov 08:25PM:
Re: Blade servers
Bah, it's just HD to the BDU, if you want to maintain the same bits-per-pixel of your current feed, (2.5 Mbps at presumably 704x480) then 1280x720 is only 6.8 Mbps.
reply
milnoc @ 22nd Nov 08:39PM:
Re: Blade servers
Does that mean I'll have to UPGRADE THE SERVERS AGAIN?!? AIEEEE!!!
Am I sounding like a penny-pinching boss yet? :D
reply
Guspaz @ 22nd Nov 09:38PM:
Re: Blade servers
No, just the bandwidth :p
reply
nigrunze @ 22nd Nov 10:37PM:
Re: Blade servers
said by milnoc :
Does that mean I'll have to UPGRADE THE SERVERS AGAIN?!? AIEEEE!!!
Am I sounding like a penny-pinching boss yet? :D
And who is going to pay for these upgrades?
Sound familiar?
reply
InvalidError @ 22nd Nov 11:06PM:
Re: Blade servers
said by milnoc :
It might be disheartening, but that's the way it has to be.
My greatest potential audience reach is still with the cable television audience. And the cable companies won't appreciate me undercutting their business by offering the general public an on-line broadcast feed that's just as good as their cable broadcast feed.
Discovery Channel makes a bunch of their in-house shows available online a week after the first air-date, presumably for that same reason.
Delaying the online stream by several days (or at least until after the BDU "live-cast") seems like a great approach if you want to limit bandwidth (most people who want to see a show and have cable that carries the channel will likely try that first) and maximize value to BDUs (online streaming is only competing with re-re-runs)
The low-def online simultaneous stream would only be warranted for stuff with ephemeral importance like news.
reply
milnoc @ 23rd Nov 12:17AM:
Re: Blade servers
I'll start with the low-res. streams first, then migrate to higher res. deferred streams later. I can do the low-res live streams right away with very few changes to the system as I've already demonstrated during the last week.
The VOD type deferred streams however will require a bit more time to implement in order to properly automate the entire process, keeping ONLY the material that's authorized for deferred redistribution. It's all distribution contract issues at this point.
One step at a time.
said by nigrunze :
And who is going to pay for these upgrades?
Sound familiar?
It sounds very different when you're on the other side of the fence. :)
EDIT: I'll be adding some new material at the end of the day, which will include our second third-party contribution, this time from Northern Green Lifestyles TV!
Most of the ephemeral films will be removed since their presence could become a problem. They're not being presented in the proper context, and they can easily be misinterpreted. However, they will return one day in a show called "Ephemeral", where a panel will view the films and discuss their sociological impact and their relevance in today's modern society.
Gotta call Carolina ETV this afternoon. With a bit of luck, I might be able to include another third-party contribution in the test broadcast.
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might crash.
Main feed: »38.104.152.86:11084 Auxiliary feed: »38.104.152.86:11089
reply
milnoc @ 23rd Nov 06:13PM:
Re: Blade servers
The schedule for the next couple of days to be launched at 6:30 PM.
setup programming input "/repository/programming/Democracy Now/dn2009-1123.mp4"
setup programming input "/repository/programming/Documentaries/Northern Green Lifestyles TV/NEGLTV202Final.avi"
setup programming input "/repository/programming/TCPub/SSI2009HQPart1Final.avi"
setup programming input "/repository/programming/TCPub/SSI2009HQPart2Final.avi"
setup programming input "/repository/programming/TCPub/SSI2009HQPart3Final.avi"
setup programming input "/repository/programming/TCPub/SSI2009BonusRoomsHQFinal.avi"
setup programming input "/repository/programming/TCPub/NokiaN900TeaserFinal.avi"
setup programming input "/repository/programming/TCPub/AsusEeePC900HQFinal.avi"
setup programming input "/repository/programming/TCPub/AsusEeePC900HAReviewHQFinal.avi"
setup programming input "/repository/programming/TCPub/AsusEeePC1000HEReviewHQFinal.avi"
setup programming input "/repository/programming/TCPub/CanonHF200SanyoVPCTH1ShootOutFinal.avi"
setup programming input "/repository/programming/Shorts/Oddball/Cliche Family in Televisionland.mp4"
setup programming input "/repository/programming/Movies/Theatrical/My Man Godfrey.mp4"
setup programming input "/repository/programming/Movies/Theatrical/Renfrew of the Royal Mounted.mp4"
setup programming input "/repository/programming/Movies/Theatrical/Suddenly.mp4"
setup programming input "/repository/programming/Movies/Theatrical/Night of the Living Dead.mp4"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand1Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand2Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand3Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand4Final.avi"
setup programming input "/repository/programming/TCPub/PaxtonBachmanBand5Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart1Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart2Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart3Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart4Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart5Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart6Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart7Final.avi"
setup programming input "/repository/programming/TCPub/YATSLondonParisPart8Final.avi"
setup programming input "/repository/programming/TCPub/EuropeanVignette1Final.avi"
setup programming input "/repository/programming/TCPub/EuropeanVignette2Final.avi"
setup programming input "/repository/programming/TCPub/EuropeanVignette3Final.avi"
It's now time for me to dive into VLC's source code and fix its various annoyances. I intend to find and resolve the problem with the videos not transitioning properly. Chances are it's something really stupid. I can feel it.
I've already designed and programmed automated traffic control and truck weighing systems from scratch under both OS/2 and Windows. I'm used to complex code. :)
--
Watch my future television channel's beta test streams! Open one of the following links in Windows Media Player. Warning, might crash.
Main feed: »38.104.152.86:11084 Auxiliary feed: »38.104.152.86:11089
reply
Thank you for using lo-fi dslreports.com - report bugs
© 99-2009 silver matrix LLC
