New Workstation Build

[Luke Boyce]

So I've been perusing these forums for the past few days trying to collect the information necessary to figure out the components needed to build a new workstation. I've used Mac Pro's my entire career thus far, but my 2008 Mac Pro is showing it's age and now that I've fully moved from FCP to Adobe, I'm not locked into the Apple Infrastructure anymore, which means I realize that I can really put together a serious machine that will give me more bang for my buck. Not that I don't love Apple, but regardless of when and if new Mac Pro's are released, I'd just like to have more options regarding expandability, especially in the GPU department.

So using the information mostly offered by Jeff Kilgore on these threads, I listed what I think seems to be a pretty decent system, and just wanted any advice regarding it or any suggestions for something I may have forgotten about. Also thought a thread pulling in some of the current information spread out through other threads may help people in a similar situation (unless there's already another thread like this that I may have missed, which is possible).

In my case, we're trying to avoid a Red Rocket. I know that if I want smooth 4k playback I should just get one, but, the pricing just doesn't fit in my budget. I'm trying to work under the $9,000 range as much as possible so it doesn't work, and I think I can get along fine without it. At least, that's what I gather. I also don't deal much in color grading apps. I've never touched Resolve and probably won't in the near future. I'm interested in Speedgrade, but generally I stick mostly to Premiere and After Effects. My production company mostly produces corporate stuff and films. For our corporate stuff we just do basic color correction and for our films we tend to have it done professionally after the edit. Up till now, I haven't had a CUDA enabled card in my system so since moving to Adobe I haven't been able to take advantage of the Mercury Playback Engine. And having worked most of my life in Final Cut, I haven't had the need to have tons and tons of RAM. That's changed now. I edit native R3D's (mostly 4k) in Premiere and do motion graphics in After Effects, but I don't have good knowledge of GPU's and that stuff. Honestly, I'm a creative director so I don't really even have any experience building PC's (although I built my own HTPC at home, but that was a fairly simple machine I built for less than $400). So all of that is to just say, I'm an intermediate at best with this stuff. Be gentle.

So here's the setup:
1. MOBO: SuperMicro Super X9DAi Motherboard
PRICE: [Included with Chassis]

2. CPU: Dual Intel Xeon E5-2687W 3.1GHz 8-Core
PRICE: $3,800

3. GPU: Nvidia GeForce GTX580
PRICE: $600

4. RAM: (2) G.SKILL Ripjaws Z Series 128GB (8 x 8GB) 600 (PC3 12800)
PRICE: $900

5. CHASSIS: SuperMicro SYS-7047A-T 4U (Pre-Wired with X9DAi Mobo)
PRICE: $1,150

6. SSD: OCZ Vertex 4 512GB SATA III MLC Internal
PRICE: $650

7. OPTICAL DRIVE: LG Black 14X Internal Blu-Ray Burner
PRICE: $120

8. OS: Windows 7 Professional 64-bit
PRICE: $130

9. MONITOR: Dell Ultrasharp U2711 27” Monitor
PRICE: $800

Couple of notes: I realize that SuperMicro is a little touchy with their RAM and they have suggestions on their site. I saw that Jeff was looking at those G.Skills so I used them mostly as a price-reference, but I wasn't sure if they would work or not.

Also, I'm extremely interested in the GTX 690 more than anything but I realize that Adobe is still working on supporting the Keplar architecture on CS6, so I wasn't sure if I should count on it or not. I noticed that Jeff had one on order, but haven't noticed any comments that he had it yet and had tested it. If I wait on anything, I think it would be to evaluate the potential of using that card, because at that price-point ($1k-$1.5k) I definitely would want to get it in place of the gtx580. Would anybody confirm that's wise? Or at least the 680? I know that's generally in the same boat as the 690, but having not used any CUDA cards, I don't know if the boost from 500 CUDA cores to the 1500 or even 3000 would be significant (common sense is telling me that it would).

P.S. I'm also interested in a RAID config with the hot-swap bays on the chassis. Correct me if I'm wrong, but I can create an internal raid using the X9DAi. Is that wise working with 4k files or should I definitely plan on getting and adapter card?

So anyway, thoughts, comments, concerns. I apologize in advance if I've wasted anybody's time with this thread. Thanks

[Richard Tran]

So I've been perusing these forums for the past few days trying to collect the information necessary to figure out the components needed to build a new workstation. I've used Mac Pro's my entire career thus far, but my 2008 Mac Pro is showing it's age and now that I've fully moved from FCP to Adobe, I'm not locked into the Apple Infrastructure anymore, which means I realize that I can really put together a serious machine that will give me more bang for my buck. Not that I don't love Apple, but regardless of when and if new Mac Pro's are released, I'd just like to have more options regarding expandability, especially in the GPU department.

So using the information mostly offered by Jeff Kilgore on these threads, I listed what I think seems to be a pretty decent system, and just wanted any advice regarding it or any suggestions for something I may have forgotten about. Also thought a thread pulling in some of the current information spread out through other threads may help people in a similar situation (unless there's already another thread like this that I may have missed, which is possible).

In my case, we're trying to avoid a Red Rocket. I know that if I want smooth 4k playback I should just get one, but, the pricing just doesn't fit in my budget. I'm trying to work under the $9,000 range as much as possible so it doesn't work, and I think I can get along fine without it. At least, that's what I gather. I also don't deal much in color grading apps. I've never touched Resolve and probably won't in the near future. I'm interested in Speedgrade, but generally I stick mostly to Premiere and After Effects. My production company mostly produces corporate stuff and films. For our corporate stuff we just do basic color correction and for our films we tend to have it done professionally after the edit. Up till now, I haven't had a CUDA enabled card in my system so since moving to Adobe I haven't been able to take advantage of the Mercury Playback Engine. And having worked most of my life in Final Cut, I haven't had the need to have tons and tons of RAM. That's changed now. I edit native R3D's (mostly 4k) in Premiere and do motion graphics in After Effects, but I don't have good knowledge of GPU's and that stuff. Honestly, I'm a creative director so I don't really even have any experience building PC's (although I built my own HTPC at home, but that was a fairly simple machine I built for less than $400). So all of that is to just say, I'm an intermediate at best with this stuff. Be gentle.

So here's the setup:
1. MOBO: SuperMicro Super X9DAi Motherboard
PRICE: [Included with Chassis]

2. CPU: Dual Intel Xeon E5-2687W 3.1GHz 8-Core
PRICE: $3,800

3. GPU: Nvidia GeForce GTX580
PRICE: $600

4. RAM: (2) G.SKILL Ripjaws Z Series 128GB (8 x 8GB) 600 (PC3 12800)
PRICE: $900

5. CHASSIS: SuperMicro SYS-7047A-T 4U (Pre-Wired with X9DAi Mobo)
PRICE: $1,150

6. SSD: OCZ Vertex 4 512GB SATA III MLC Internal
PRICE: $650

7. OPTICAL DRIVE: LG Black 14X Internal Blu-Ray Burner
PRICE: $120

8. OS: Windows 7 Professional 64-bit
PRICE: $130

9. MONITOR: Dell Ultrasharp U2711 27” Monitor
PRICE: $800

Couple of notes: I realize that SuperMicro is a little touchy with their RAM and they have suggestions on their site. I saw that Jeff was looking at those G.Skills so I used them mostly as a price-reference, but I wasn't sure if they would work or not.

Also, I'm extremely interested in the GTX 690 more than anything but I realize that Adobe is still working on supporting the Keplar architecture on CS6, so I wasn't sure if I should count on it or not. I noticed that Jeff had one on order, but haven't noticed any comments that he had it yet and had tested it. If I wait on anything, I think it would be to evaluate the potential of using that card, because at that price-point ($1k-$1.5k) I definitely would want to get it in place of the gtx580. Would anybody confirm that's wise? Or at least the 680? I know that's generally in the same boat as the 690, but having not used any CUDA cards, I don't know if the boost from 500 CUDA cores to the 1500 or even 3000 would be significant (common sense is telling me that it would).

P.S. I'm also interested in a RAID config with the hot-swap bays on the chassis. Correct me if I'm wrong, but I can create an internal raid using the X9DAi. Is that wise working with 4k files or should I definitely plan on getting and adapter card?

So anyway, thoughts, comments, concerns. I apologize in advance if I've wasted anybody's time with this thread. Thanks

Raid Card?

[Brian Merlen]

not to rain on ur plan, but wouldn't it be far easier to just buy a new 12 core mac and keep using all your same accessories and software? if your set on going the pc route far be it from me to stop you, but i figured all ur software is mac based right now anyway...

[Luke Boyce]

Raid Card?

I'm still confused about how necessary it is. I keep getting mixed reactions. Can I set one up just using the X9DAi? What are the benefits of using a card that would justify the $800-$1,000 hike? And I'm asking cause I seriously have no idea.

[Luke Boyce]

not to rain on ur plan, but wouldn't it be far easier to just buy a new 12 core mac and keep using all your same accessories and software? if your set on going the pc route far be it from me to stop you, but i figured all ur software is mac based right now anyway...

Well, my software isn't really mac-based at the moment. I'm planning on getting CS6 right away but it doesn't have to be mac and that's pretty much the majority of my software needs right there. And at the moment I don't really have any mac-specific accessories that I'm using. If I was going to stay the Mac route I'd most likely just wait and get the new ones whenever they're announced (I'm pretty confident they will be by June). But honestly, the only thing that would keep me in the Mac game would be the familiarity with the OS, but I just don't consider it significant enough to ignore the drastically more upgradable route of a PC workstation.

Great example is the GPU issue. I've wanted to upgrade my GPU for a few months now, to take advantage of the Mercury Engine in Premiere. When looking it up the lack of options other than a Quadro 4000 is just pathetic. Without having to make sure you have this, and make sure you do that with another card is annoying. I'd prefer a situation where, I could get a gtx 580 now, and it works. And then in 6 months if I want, get a 690 or whatever is available without any headaches. I mean, why shouldn't I have that option, just because I've used OS X most of my life?

[Richard Tran]

http://thessdreview.com/our-reviews/highpoint-2720sgl-rocketraid-controller-review-amazing-3gbs-recorded-with-8-crucial-c400-ssds/
this for speed,
and external RAID 1 enclosures for security?

The Card itself is inexpensive.

[Luke Boyce]

http://thessdreview.com/our-reviews/highpoint-2720sgl-rocketraid-controller-review-amazing-3gbs-recorded-with-8-crucial-c400-ssds/
this for speed,
and external RAID 1 enclosures for security?

The Card itself is inexpensive.

Interesting. I'll definitely look into that. The SuperMicro SYS-7047A-T has 8 bays in front for HDD's so we were going to probably just fill it with 7200rpm 3.5" drives.

[Jeff Kilgroe]

So I've been perusing these forums for the past few days trying to collect the information necessary to figure out the components needed to build a new workstation. I've used Mac Pro's my entire career thus far, but my 2008 Mac Pro is showing it's age and now that I've fully moved from FCP to Adobe, I'm not locked into the Apple Infrastructure anymore, which means I realize that I can really put together a serious machine that will give me more bang for my buck. Not that I don't love Apple, but regardless of when and if new Mac Pro's are released, I'd just like to have more options regarding expandability, especially in the GPU department.

Aside from GPU selection or the ability to build monster systems with 4 or 8 CPUs, the Mac really doesn't lack in power like a lot of people think it does. Of course, they have have not announced a new system yet using the new Xeon E5-2600 series CPUs. Then again, most of the major PC vendors haven't either and of the "big guys", HP is the only one shipping them so far. That's been another hot topic around here lately, but we're all just guessing as to what Apple's next move will be.


In my case, we're trying to avoid a Red Rocket. I know that if I want smooth 4k playback I should just get one, but, the pricing just doesn't fit in my budget.

Avoiding the Rocket may be a good idea, it may not. You can get by with a lesser system for R3D workflow if you have a Rocket. And if you omit the Rocket, you're going to need a much more powerful system to accomplish the same thing. Of course, building a very powerful system up front still leaves you with the option to add the Rocket down the road.


I'm trying to work under the $9,000 range as much as possible so it doesn't work, and I think I can get along fine without it. At least, that's what I gather. I also don't deal much in color grading apps. I've never touched Resolve and probably won't in the near future. I'm interested in Speedgrade, but generally I stick mostly to Premiere and After Effects. My production company mostly produces corporate stuff and films. For our corporate stuff we just do basic color correction and for our films we tend to have it done professionally after the edit. Up till now, I haven't had a CUDA enabled card in my system so since moving to Adobe I haven't been able to take advantage of the Mercury Playback Engine. And having worked most of my life in Final Cut, I haven't had the need to have tons and tons of RAM. That's changed now. I edit native R3D's (mostly 4k) in Premiere and do motion graphics in After Effects, but I don't have good knowledge of GPU's and that stuff. Honestly, I'm a creative director so I don't really even have any experience building PC's (although I built my own HTPC at home, but that was a fairly simple machine I built for less than $400). So all of that is to just say, I'm an intermediate at best with this stuff. Be gentle.

Building a system isn't that difficult, but if you don't have much experience a system like these current dual-Xeon configurations can be rather daunting. Just hunting down the parts can be a significant task in itself. So I have to ask if you've considered buying a system like the HP Z820? I received one here today and there's a lot about it to like. There is another thread around here, where I compared my Z820 configuration to a similar build using the SuperMicro X9DAi motherboard. The HP did come at a premium -- in the case of a pretty extreme build, we're looking at about a $1000 premium. But the build quality is rather good and it's a well thought out system. Full 3-year on-site warranty and you don't have to spend all your time hunting down parts, assembling a system and testing it.

Anyway, on to your parts list...



So here's the setup:
1. MOBO: SuperMicro Super X9DAi Motherboard
PRICE: [Included with Chassis]


Yep, probably the best all-around E5-2600 motherboard right now. The ASUS board looks great and I'm hearing good things about it. However it has fewer PCIe and memory slots, so it's geared toward smaller configurations. The EVGA SR-X looks interesting, but the PCIe configuration is somewhat mysterious, as is the 12 DIMM slots. I'm waiting for some people to start posting in-depth reviews before I form an opinion. However, I'm probably done buying/building systems with this generation of Xeons until the Ivy Bridge Xeons ship sometime in about 10 to 14 months from now.

The X9DAi motherboard goes for about $450 and are still somewhat scarce. Easier to find and buy pre-wired into the SuperMicro chassis. This is not a bad way to do it.


2. CPU: Dual Intel Xeon E5-2687W 3.1GHz 8-Core
PRICE: $3,800

They're currently the fastest in the series. And they also run the hottest and suck up 150W each. HP won't sell them without their liquid coolers, so you need to have adequate ventilation and good coolers installed for these.


3. GPU: Nvidia GeForce GTX580
PRICE: $600

Get the EVGA Classified Ultra model. It's the most hot-rod version of the card out there and also runs perfectly stable. Love this card. $529.99 at Amazon right now. :) Or $20 more at NewEgg, but they also offer a bundle for an additional $109 you get Adobe Lightroom 4 with it. Not a bad deal if you need Lightroom. I have the Z820 up and running, but haven't got too far with it just yet. Put a Quadro 6000 card in it for now. Will also test with the above-mentioned GTX580 and a couple other GPUs and various combinations. I haven't dug into Adobe CS6 much, will do so this weekend I hope.



4. RAM: (2) G.SKILL Ripjaws Z Series 128GB (8 x 8GB) 600 (PC3 12800)
PRICE: $900

Don't do it. I was considering this as a possibility in the beginning just because I couldn't find any of the approved RAM anywhere and these were relatively affordable as a temporary solution -- G.Skill usually has pretty good stuff. I recommend getting the Micron or Samsung RAM that is approved as per the SuperMicro web site. You can also order the proper RAM for this system now through Crucial or NewEgg has modules from both WinTec and Kingston that are compatible. You're looking at about $130 or so per 8GB module.


6. SSD: OCZ Vertex 4 512GB SATA III MLC Internal
PRICE: $650

It's fast. :) Write speeds are still a little underwhelming. Similar to the Samsung 830 SSD, but the read speeds are consistently saturating the 6Gbps SATA connection at about 550MB/s. Make sure to connect with a good quality cable and you don't need to buy an additional 2.5" to 3.5" adapter bracket as OCZ includes one with the SSD.


7. OPTICAL DRIVE: LG Black 14X Internal Blu-Ray Burner
PRICE: $120

OK.


8. OS: Windows 7 Professional 64-bit
PRICE: $130

Yep...


9. MONITOR: Dell Ultrasharp U2711 27” Monitor
PRICE: $800

Decent monitor at a good price. I own several Dell monitors...

Couple of notes: I realize that SuperMicro is a little touchy with their RAM and they have suggestions on their site. I saw that Jeff was looking at those G.Skills so I used them mostly as a price-reference, but I wasn't sure if they would work or not.


Also, I'm extremely interested in the GTX 690 more than anything but I realize that Adobe is still working on supporting the Keplar architecture on CS6, so I wasn't sure if I should count on it or not. I noticed that Jeff had one on order, but haven't noticed any comments that he had it yet and had tested it. If I wait on anything, I think it would be to evaluate the potential of using that card, because at that price-point ($1k-$1.5k) I definitely would want to get it in place of the gtx580. Would anybody confirm that's wise? Or at least the 680? I know that's generally in the same boat as the 690, but having not used any CUDA cards, I don't know if the boost from 500 CUDA cores to the 1500 or even 3000 would be significant (common sense is telling me that it would).

Don't have the 690 yet... Not sure when I'll get it. It will get here when it gets here and hopefully the drivers won't be in such a beta state when it arrives. I hear they're pretty flaky right now. Although there are more CUDA cores, the speed doesn't translate linearly. Kepler is a new architecture with a different design philosophy. Using many more smaller and less powerful cores to do the work. Some operations are going to be significantly faster than the previous Fermi architecture. Others not so much, some even slower.


P.S. I'm also interested in a RAID config with the hot-swap bays on the chassis. Correct me if I'm wrong, but I can create an internal raid using the X9DAi. Is that wise working with 4k files or should I definitely plan on getting and adapter card?

Yes to the internal RAID. I would recommend using RAID-0 for speed for your primary working array. But you will want a good backup system in place. Even if you add-in another RAID card and go to RAID-5 or better, never consider that completely reliable. Once again, that's a whole other discussion... One that seems to rear it's head on these forums every now and then.

[Fabian Benninghoven]

Maybe you are interested in other MoBos than Supermicro... today i received this email:

Dear EVGA Customer,

EVGA is happy to announce the product that you requested to be notified is now available.

Please click on the link below to make your purchase.

http://www.evga.com/products/moreinfo.asp?pn=270-SE-W888-KR

If you have any questions you can email us at sales@evga.com (sales@evga.com?subject=EVGA%20Product%20Notificat ion%20Question)
Thank you,

EVGA Sales Team

[Luke Boyce]

Aside from GPU selection or the ability to build monster systems with 4 or 8 CPUs, the Mac really doesn't lack in power like a lot of people think it does. Of course, they have have not announced a new system yet using the new Xeon E5-2600 series CPUs. Then again, most of the major PC vendors haven't either and of the "big guys", HP is the only one shipping them so far. That's been another hot topic around here lately, but we're all just guessing as to what Apple's next move will be. Ya, don't get me wrong, my Mac Pro's have been incredible. And we'll continue to use our 2008 as another editing station for smaller form stuff like DSLR shoots, etc. And I'm probably going to wait to buy anything until the new one's have been released, or at least until after WWDC, whichever comes first, just to make sure, but generally I'm just wanting to go the custom-PC route for the sake of future-flexibility.


So I have to ask if you've considered buying a system like the HP Z820? I received one here today and there's a lot about it to like. There is another thread around here, where I compared my Z820 configuration to a similar build using the SuperMicro X9DAi motherboard. The HP did come at a premium -- in the case of a pretty extreme build, we're looking at about a $1000 premium. But the build quality is rather good and it's a well thought out system. Full 3-year on-site warranty and you don't have to spend all your time hunting down parts, assembling a system and testing it. I've very briefly looked at the Z820 after seeing you mention it more than once. I need to take a deeper look at it, honestly, but it's certainly not out of the question. In the end, it'll probably all come down to cost. The "hunting" part of the process I rather like. :smile:


They're currently the fastest in the series. And they also run the hottest and suck up 150W each. HP won't sell them without their liquid coolers, so you need to have adequate ventilation and good coolers installed for these. Ah, not something I considered. Would it be more prudent, then, to go with something like the E5-2690's then, if I'm not planning on going with liquid cooling, or is there something you'd suggest here alternatively?


Get the EVGA Classified Ultra model. It's the most hot-rod version of the card out there and also runs perfectly stable. Love this card. $529.99 at Amazon right now. :) Or $20 more at NewEgg, but they also offer a bundle for an additional $109 you get Adobe Lightroom 4 with it. Not a bad deal if you need Lightroom. I have the Z820 up and running, but haven't got too far with it just yet. Put a Quadro 6000 card in it for now. Will also test with the above-mentioned GTX580 and a couple other GPUs and various combinations. I haven't dug into Adobe CS6 much, will do so this weekend I hope. Awesome. Thanks for the suggestion. I will most definitely do that. And I very much look forward to the results of your testing.


Don't do it. I was considering this as a possibility in the beginning just because I couldn't find any of the approved RAM anywhere and these were relatively affordable as a temporary solution -- G.Skill usually has pretty good stuff. I recommend getting the Micron or Samsung RAM that is approved as per the SuperMicro web site. You can also order the proper RAM for this system now through Crucial or NewEgg has modules from both WinTec and Kingston that are compatible. You're looking at about $130 or so per 8GB module. That's certainly what I was expecting to hear. Which means, for cost-purposes, I may have to go the 64gb route for the time being, then, so I don't double my cost on RAM, and then upgrade later.


Don't have the 690 yet... Not sure when I'll get it. It will get here when it gets here and hopefully the drivers won't be in such a beta state when it arrives. I hear they're pretty flaky right now. Although there are more CUDA cores, the speed doesn't translate linearly. Kepler is a new architecture with a different design philosophy. Using many more smaller and less powerful cores to do the work. Some operations are going to be significantly faster than the previous Fermi architecture. Others not so much, some even slower. Sounds like the 580 is the way to go at the moment and I won't be disapointed. Good to hear. Granted, I'm coming from using NO CUDA cores, so anything will probably impress me speed-wise. But this, again, is one of the main reasons I'm considering going this route. It would feel great to know that, should a card like that end up being pretty amazing, I could always just add it in 8 months or so. Or for that matter, use the 580 for now and get whatever monster they release around NAB next year.

All in all...holy crap. Thank you so much for being insanely brilliant and so willing to offer up your expertise. Knowing this board is here and people like you are on it and generously offer your help makes jumping into this kind of thing not as scary as it could be. My appreciation is endless. Thank you so much.

[Jeff Kilgroe]

YI've very briefly looked at the Z820 after seeing you mention it more than once. I need to take a deeper look at it, honestly, but it's certainly not out of the question. In the end, it'll probably all come down to cost. The "hunting" part of the process I rather like. :smile:

I've had a Z820 in my possession here for about a day now. Overall I like it a lot. HP did alright with the PCIe slot arrangement, but I've encountered one issue I'm not too happy about -- the lowest PCIe X16 slot is perfect for accepting double-width cards as the adjacent slot (the bottom one) is a standard PCI 32bit legacy slot. Unfortunately, the Quadro 6000 (and also the GTX580 or similar) won't fit there because of all the other stuff running along the bottom of the motherboard. Seems like a real goof to me. Oh well... The GTX580 or 680 HydroCopper versions from EVGA would fit like a charm, but I would have to run an external reservoir and radiator as there's no way to incorporate it into the internal system without sacrificing a drive bay. I have all the bays filled and have even mounted my system drive, the SSD, in the gap above the top 5.25" drive bay. So that gives me the SSD, the Blu-Ray writer, and 6 x 2TB HDD's for workspace. :)


Ah, not something I considered. Would it be more prudent, then, to go with something like the E5-2690's then, if I'm not planning on going with liquid cooling, or is there something you'd suggest here alternatively?

2690's are a consideration, they seem to do just fine with air cooling. In the end, price will be similar. A good liquid system or, self-contained liquid coolers like what HP is using, come at a premium so that just about offsets the lower cost of the 2687W CPUs. Performance is going to be mostly similar, but the 2687's do score better in intensive tests and benchmarks.


That's certainly what I was expecting to hear. Which means, for cost-purposes, I may have to go the 64gb route for the time being, then, so I don't double my cost on RAM, and then upgrade later.

I have 64GB in both the HP system and the other SuperMicro based system I've put together. At least for the moment until my other RAM arrives (today). 64GB is a good amount. But for intense rendering where I can maximize all the cores/threads on this system, it's a bit tight. I would recommend the Samsung or Micron DIMMs if you can find them, it's a pain. I had a round-about way of getting them from Micron. There's these modules here by Kingston, which should also work just fine, but they're a bit slower at CAS-11. Which is what the HP system is using. They're HP-labeled, but I'm positive these are the same modules in my Z820. http://www.newegg.com/Product/Product.aspx?Item=N82E16820239015


Sounds like the 580 is the way to go at the moment and I won't be disapointed. Good to hear. Granted, I'm coming from using NO CUDA cores, so anything will probably impress me speed-wise. But this, again, is one of the main reasons I'm considering going this route. It would feel great to know that, should a card like that end up being pretty amazing, I could always just add it in 8 months or so. Or for that matter, use the 580 for now and get whatever monster they release around NAB next year.

Only card I would potentially recommend over the 580 right now would be the Quadro 6000. It's going to be slower for some stuff, especially performance visuals and pushing textures around. On the other hand, even with 3GB you can bottleneck the card with that RAM amount if you're really loading it up with a lot of data. Overall it's better tuned and runs silky smooth in various 3D apps. I like it, but it's pricey. I'll be doing comparison testing between the Quadro 6000 and GTX580 in the HP system this weekend. I still have to get all my software installed though, I just finished installing all the HDDs a few minutes ago and out of time for today on that project.

[Richard Tran]

Jeff,

What would you recommend for a good quality cable?

Any thoughts on the HighPoint raid card?
I think you mentioned you recommend ATTO only?

-R


It's fast. :) Write speeds are still a little underwhelming. Similar to the Samsung 830 SSD, but the read speeds are consistently saturating the 6Gbps SATA connection at about 550MB/s. Make sure to connect with a good quality cable and you don't need to buy an additional 2.5" to 3.5" adapter bracket as OCZ includes one with the SSD.

[Jeff Kilgroe]

Cable for what? The SSD? I'm using the OKGEAR or StarTech 6G/SATA-3 rated cables.

HighPoint RAID cards are good. On the Mac I tend to stick with ATTO. Areca are also great, but I prefer the ATTO since they interface with LTO drives better and have proven bullet-proof.

On the PC, ATTO, Areca, HighPoint, LSI are all good. Some can be recommended more than others depending on which devices you intend to hook up or what configurations you're looking at. If you will be going all SATA drives and want great speed for a low price, the HighPoint cards are tough to beat.

[Richard Tran]

Cable for what? The SSD? I'm using the OKGEAR or StarTech 6G/SATA-3 rated cables.

HighPoint RAID cards are good. On the Mac I tend to stick with ATTO. Areca are also great, but I prefer the ATTO since they interface with LTO drives better and have proven bullet-proof.

On the PC, ATTO, Areca, HighPoint, LSI are all good. Some can be recommended more than others depending on which devices you intend to hook up or what configurations you're looking at. If you will be going all SATA drives and want great speed for a low price, the HighPoint cards are tough to beat.


Yup, SSD.

Thank you for all the advice.

[Alex Mills]

quick question regarding raids, I have a mac pro going to a 8tb proavio raid through a rocket raid card, if I wanted to swich everything to a pc, how would I move the raid, what are my options?

[Frank Cueto]

Luke & Jeff, any updates on your new workstations? I am particularly interested in cinebench scores.

Thanks!

[Alex Mills]

quick question regarding raids, I have a mac pro going to a 8tb proavio raid through a rocket raid card, if I wanted to swich everything to a pc, how would I move the raid, what are my options?

[Richard Tran]

quick question regarding raids, I have a mac pro going to a 8tb proavio raid through a rocket raid card, if I wanted to swich everything to a pc, how would I move the raid, what are my options?

I think you'll have to reformat the whole thing.

Since you're changing from HFS, to NTFS..

[Frank Cueto]

Or maybe macdrive could be of help?

[Jeff Kilgroe]

Something like MacDrive should work, but it may not. Some of these RAID cards may change the way they address drives in a RAID volume depending on whether they are in a Mac or PC environment. Shouldn't happen, but we're talking about EFI / BIOS differences and driver differences between two platforms. So, to play it safe, back up all the data before you move any hardware around. You should do that no matter what anyway...

Other than that, after you have a backup, you should be able to just move the RAID from the Mac to the PC. I would think you'll definitely want to reformat as NTFS and re-copy your data unless you have a desire to move it back to a Mac in the near future. Accessing HPFS volumes via third-party software always carries an elevated level of risk and it can impact the possibility of getting the best possible performance.

[Frank Cueto]

Jeff, how about some cinebench scores on the HPz820?

[Jeff Kilgroe]

Jeff, how about some cinebench scores on the HPz820?

Cinebench is starting to show that it's not a great benchmark for these new systems. The test isn't intensive enough or long-running enough to kick in the turbo on the CPUs. :)

The Z820 system is giving me a render score of 25.31 and 84.98 on the render with the Quadro 6000 GPU.

But like I said, it's not capitalizing on the CPU abilities... I just compared the base numbers on my Z820 for now until I really get to dig into tweaking everything. On my nearly identical build using the SuperMicro motherboard and the same CPUs, I managed a render score of 31.09 in Cinebench. But I had to crank up the minimum CPU in the Windows settings as high as it would go (default is 5%) and I had to run another render in Modo to get all 32 threads firing at full speed then started the Cinebench render test a fraction of a second before the modo render completed! So yeah, I kinda cheated... But it shows Cinebench to be a poor way to benchmark these new systems.

[Frank Cueto]

If its not asking too much, how about doing the same cheat on the HP box.

I am Incredibly torn between the HP/Home built and a TOP END BOXX, and I am suffering of paralysis by analysis, if u you know what i mean.

[Jeff Kilgroe]

I'll give it a try after I get everything set up on the HP box. I have an issue to iron out with the Vertex 4 SSD I've installed. It's giving me some erratic performance and the system likes to hang and reboot itself one every 2 or 3 times it wakes from sleep. I think that is related to the SSD as it didn't do it when booting from the stock HDD.

FWIW, the SuperMicro build was giving me a 25.10 running the Render test through a few times. So the HP box seems to be tuned well.

Based on the components in the BOXX system for the price I would steer you away from them. They're using an Arima motherboard, from what I can tell, which are usually pretty good, but it's just not at the same level as the SuperMicro X9DAi or their similar offerings or the HP. You can only install half as much RAM and you have fewer PCIe configuration options. It's more comparable to doing a self-build with the ASUS dual E5 motherboard. On that note, I'll say that I'm hearing good things about the ASUS board -- if you don't need all the PCIe slots or the 16 DIMM sockets. It brings some overclocking options to the table as well. Not much as the Xeons are multiplier locked and voltage limited, but there is a little freedom to play there that you don't get with the SuperMicro boards. Not sure what the EVGA SR-X is going to offer. I don't know anyone who has one yet.

[Olivier Madar]

Hi Jeff,

I have to choose beetwen Super micro and HP and as you are the only one personn that have both, i ask you that question. I know that the Supermicro is bigger and better but I am afraid of the noise because I can't put the computer behind the wall. So, what is the difference in noise between the two ?


Olivier

[Jeff Kilgroe]

I wouldn't say the SuperMicro is better. Only real difference is the PCI slot configuration and since you're assembling yourself, you can place it in as big of a case as you would like. The HP system is very quiet. Comparable to a Mac Pro, the only fan that I audibly hear ramp up under load so far is the one on the Quadro 6000.

The SuperMicro build on my end is noisier at the moment, but I can make it quieter by swapping out fans and whatnot. Not sure what else to say on that, it's as quiet as you can build it. If you want the quietest option, you will want to go liquid cooled.

[Clay Glendenning]

Jeff,

Thanks again for being such an incredibly valuable asset to the forums. I bet you are a little tired of the endless questions. I think the timing just so happens to coincide with many of us needing to make large hardware upgrades and studio changes during a very chaotic, for lack of a better word, time in the software and hardware arena for production.

I'll PM you if I need to, before I fill this thread with the build data. I'm building out a Z820, and soloing all of the research. We own a 4-slot double wide cubix with a RR and GTX 570 (space left to add a few things down the road, and we can move around the aforementioned in the new proposed machine). This gets used on a Mac Pro 3,1 which will find use as another less important machine.

We plan on doing a business lease for the machine, I'll find out today if I'm inhibited by that in any big way. My biggest questions: Are there any known issues with the hard drives that HP has as options during the build? I am debating between a 1x256gb for the Win7 OS & Apps, then 3x300gb 15k SAS drives in a RAID 5 array for working drives. Those are all HP drives, and I am considering replacing those working raided drives with SSDs. I can have the work drives come pre-raided with an LSI MegaRAID SAS 9260-8i. That is convenient, but I could prefer an ATTO card just because I have seen them perform well over the years.

The internal HD setup and raid configuration continues to be my primary concern. We'll be doing the E5 2687 3.1ghz config with 64gb RAM (I'm on the fence and may go 128gb). We might end up purchasing one of the OWC Jupiter offerings for external fast data, once more details/pricing are known. LTO setup for backups, most likely.

Cheers and many thanks.

[Luke Boyce]

I've had a Z820 in my possession here for about a day now. Overall I like it a lot. HP did alright with the PCIe slot arrangement, but I've encountered one issue I'm not too happy about -- the lowest PCIe X16 slot is perfect for accepting double-width cards as the adjacent slot (the bottom one) is a standard PCI 32bit legacy slot. Unfortunately, the Quadro 6000 (and also the GTX580 or similar) won't fit there because of all the other stuff running along the bottom of the motherboard. Seems like a real goof to me. Oh well... The GTX580 or 680 HydroCopper versions from EVGA would fit like a charm, but I would have to run an external reservoir and radiator as there's no way to incorporate it into the internal system without sacrificing a drive bay. I have all the bays filled and have even mounted my system drive, the SSD, in the gap above the top 5.25" drive bay. So that gives me the SSD, the Blu-Ray writer, and 6 x 2TB HDD's for workspace. :)I did a custom configuration of the Z820 with the 2687W's and Liquid cooling and it was QUITE a bit more expensive than I was expecting. Not sure it's worth the extra money to be honest, even though it is very tempting. Just don't have the budget to go that route, unfortunately.


2690's are a consideration, they seem to do just fine with air cooling. In the end, price will be similar. A good liquid system or, self-contained liquid coolers like what HP is using, come at a premium so that just about offsets the lower cost of the 2687W CPUs. Performance is going to be mostly similar, but the 2687's do score better in intensive tests and benchmarks.I think I'll definitely still go with the 2687w's and just get a liquid cooler. Do you have any suggestions by chance? Never had to look into them, to be honest. That's pretty new ground for me. Would something like this (http://www.newegg.com/Product/Product.aspx?Item=N82E16835181017) work? I'd have to get one for each CPU, correct?

Also, I'm now wondering if I don't want to look into a different chassis than the pre-wired Supermicro one. Are there any specifics I need to keep in mind with that. I've basically just been looking around at the E-ATX compatible cases on Newegg. From what I can tell I basically need to look into something that's ready for liquid cooling, right? Would cases along these lines work: http://www.newegg.com/Product/Product.aspx?Item=N82E16811147157 (Pretty new to this, can you tell)? And of course, I have to make sure I get a power supply as well, I realize (This? http://www.newegg.com/Product/Product.aspx?Item=N82E16817139014)


I have 64GB in both the HP system and the other SuperMicro based system I've put together. At least for the moment until my other RAM arrives (today). 64GB is a good amount. But for intense rendering where I can maximize all the cores/threads on this system, it's a bit tight. I would recommend the Samsung or Micron DIMMs if you can find them, it's a pain. I had a round-about way of getting them from Micron. There's these modules here by Kingston, which should also work just fine, but they're a bit slower at CAS-11. Which is what the HP system is using. They're HP-labeled, but I'm positive these are the same modules in my Z820. http://www.newegg.com/Product/Produc...82E16820239015That RAM doesn't look so bad in price actually. Seems I could go 128gb with those at only a few hundred dollars more.


Only card I would potentially recommend over the 580 right now would be the Quadro 6000. It's going to be slower for some stuff, especially performance visuals and pushing textures around. On the other hand, even with 3GB you can bottleneck the card with that RAM amount if you're really loading it up with a lot of data. Overall it's better tuned and runs silky smooth in various 3D apps. I like it, but it's pricey. I'll be doing comparison testing between the Quadro 6000 and GTX580 in the HP system this weekend. I still have to get all my software installed though, I just finished installing all the HDDs a few minutes ago and out of time for today on that project.Looking forward to those 6000 vs. 580 tests. I also noticed this weekend that you can actually throw a 580 in the Mac Pro with 10.7.3 and it's plug 'n play. Do you have experience with that? Are there any caveats I should know about if I wanted to just splurge at the moment and get one to spice up my current Premiere system on my 2008 Mac Pro? I read there may be issues with the 6-pin this or that. Didn't do any extensive reading, but the possibility is intriquing.

[Olivier Madar]

Thank you very, very, very much Jeff,


I will go the HP route then.



Olivier


I wouldn't say the SuperMicro is better. Only real difference is the PCI slot configuration and since you're assembling yourself, you can place it in as big of a case as you would like. The HP system is very quiet. Comparable to a Mac Pro, the only fan that I audibly hear ramp up under load so far is the one on the Quadro 6000.

The SuperMicro build on my end is noisier at the moment, but I can make it quieter by swapping out fans and whatnot. Not sure what else to say on that, it's as quiet as you can build it. If you want the quietest option, you will want to go liquid cooled.

[Jeff Kilgroe]

I did a custom configuration of the Z820 with the 2687W's and Liquid cooling and it was QUITE a bit more expensive than I was expecting. Not sure it's worth the extra money to be honest, even though it is very tempting. Just don't have the budget to go that route, unfortunately.

My current SuperMicro based build and my HP are essentially identical in spec.. The HP worked out to about $1300 more, so yes there is a premium. All things considered, it's a small price to pay if you're short on time or don't want to chase down all the parts and test and all that, plus it has the 3 year on-site warranty included. Just be sure to use the coupon codes! Without them, you won't be able to afford buying their systems. lol


That RAM doesn't look so bad in price actually. Seems I could go 128gb with those at only a few hundred dollars more.

Yeah, not too bad. I'm probably going to order some to get my HP system up to 128GB. Although I'm going to wait for the Memorial Day holiday. HP usually has coupon codes and their official modules are currently $165 each, so not much more, and I bet I can get them for the same price or less if they have a sale.

As for the liquid cooling, I don't have any recommendations at the moment. I tried both the LGA-2011 compliant cooling blocks from CoolerMaster and ASETEK and neither one would fit properly over the CPUs (not big enough to fully cover) and the Cooler Master ones wouldn't properly mount to the sockets on the SuperMicro board. Kinda weird. I never got a response from ASETEK and CoolerMaster said they are working on a solution for the new Xeons. I'm using stock air coolers at the moment and they're working OK, but noisier than I would like and the CPUs run hotter on that build than on the HP.


Looking forward to those 6000 vs. 580 tests. I also noticed this weekend that you can actually throw a 580 in the Mac Pro with 10.7.3 and it's plug 'n play. Do you have experience with that? Are there any caveats I should know about if I wanted to just splurge at the moment and get one to spice up my current Premiere system on my 2008 Mac Pro? I read there may be issues with the 6-pin this or that. Didn't do any extensive reading, but the possibility is intriquing.

I'm looking forward to doing them here in the next couple days, just been having trouble getting to it. I'm not sure about the '08 Mac Pro with a GTX580... You're kinda reaching there and I see other bottlenecks holding you back. For starters, the RAM on the system is clocked at 800Mhz and it's already the bottleneck when working at 1080p and higher in Resolve on that platform. I would also be curious as to the power abilities in the older tower and if it really could drive a 580. It works alright in a 2009/2010 tower -- I addressed it in the Ne Mac Pro thread in this forum a little bit ago. In an '08 Mac Pro, I think I would go more for a GTX460 or even scrounge up an actual Mac Edition GTX285. It's still a great card and actually has pretty darn good CUDA performance. So does the Quadro 4000 Mac Edition. Of course the GTX580 is better if you can get it up and running, but like I said, I see other bottlenecks holding you back with the older system.


Thank you very, very, very much Jeff,


I will go the HP route then.


Cool. The quietest option is to choose their liquid cooling solution -- which is two self-contained liquid-based CPU coolers.

Be sure to track down all the HP discount codes! They have a 20% off code and depending on other factors, you can often get another 3% off of that, but it doesn't look like that one is running at the moment, expired April 30. The 20% runs through July. Which at that time, they'll probably adjust pricing and issue new coupons. HP always has big coupons like that 20% to get their system prices down into the realm of normal.

[Luke Boyce]

As for the liquid cooling, I don't have any recommendations at the moment. I tried both the LGA-2011 compliant cooling blocks from CoolerMaster and ASETEK and neither one would fit properly over the CPUs (not big enough to fully cover) and the Cooler Master ones wouldn't properly mount to the sockets on the SuperMicro board. Kinda weird. I never got a response from ASETEK and CoolerMaster said they are working on a solution for the new Xeons. I'm using stock air coolers at the moment and they're working OK, but noisier than I would like and the CPUs run hotter on that build than on the HP.Hmm. Interesting. I was looking at the Corsair H80. I'll have to dig a little deeper.


I'm looking forward to doing them here in the next couple days, just been having trouble getting to it. I'm not sure about the '08 Mac Pro with a GTX580... You're kinda reaching there and I see other bottlenecks holding you back. For starters, the RAM on the system is clocked at 800Mhz and it's already the bottleneck when working at 1080p and higher in Resolve on that platform. I would also be curious as to the power abilities in the older tower and if it really could drive a 580. It works alright in a 2009/2010 tower -- I addressed it in the Ne Mac Pro thread in this forum a little bit ago. In an '08 Mac Pro, I think I would go more for a GTX460 or even scrounge up an actual Mac Edition GTX285. It's still a great card and actually has pretty darn good CUDA performance. So does the Quadro 4000 Mac Edition. Of course the GTX580 is better if you can get it up and running, but like I said, I see other bottlenecks holding you back with the older system.Good to know. Think I'll just not worry about it at the moment. Thanks for the info.


My current SuperMicro based build and my HP are essentially identical in spec.. The HP worked out to about $1300 more, so yes there is a premium. All things considered, it's a small price to pay if you're short on time or don't want to chase down all the parts and test and all that, plus it has the 3 year on-site warranty included. Just be sure to use the coupon codes! Without them, you won't be able to afford buying their systems. lolHmm. I've priced my PC build at only $8k including the 128gb of RAM and the Dell monitor, but even with the 20% off I couldn't get the HP under $10k, and that was without a monitor. Although, honestly, the PC build doesn't really scare me at all. I enjoy the process of hunting down all the parts, and I've got a friend who builds servers regularly who is willing to do the actual build for me. Just hoping to make sure I get all the right components.

Also, I'm now wondering if I don't want to look into a different chassis than the pre-wired Supermicro one. Are there any specifics I need to keep in mind with that. I've basically just been looking around at the E-ATX compatible cases on Newegg. From what I can tell I basically need to look into something that's ready for liquid cooling, right? Would cases along these lines work: http://www.newegg.com/Product/Produc...82E16811147157 (Pretty new to this, can you tell)? And of course, I have to make sure I get a power supply as well, I realize (This? http://www.newegg.com/Product/Produc...82E16817139014).

[Jeff Kilgroe]

Your new egg links are broken. :( I guess I could try plugging in the product numbers maybe.

My system thru HP was around $8300 after the 20% off and after another 3% off for preferred business customers, but that promotion expired on 4/30, the 20% is still going.. So my final price was about $8700 after tax and shipping. I got the shipping credited because I paid for 2-day and they shipped it ground. I ordered the dual 2687W CPUs (liquid cooling upgrade required), 64GB (8 x 8GB - 2CPU option), and 1 x 2TB internal HDD with the Blu-Ray writer. No video card. Looking back on it, I should have loaded up the system with 2TB HDD's. They're using Hitachi UltraStars and after the discounts, the price for those HDDs would have been nearly $50 each cheaper than NewEgg. oh well...

I pulled the Hitachi drive it shipped with and installed 6 matched WD 2TB RE4 HDD's. I've got those 6HDDs in a RAID-0 now and I'm running an OCZ Vertex4 SSD as my system drive. I mounted it via its included 3.5" adapter plate and zip-tying it off in the gap above the upper drive bays. I don't know why, but HP shipped the Blu-Ray in the middle of the 3 external bays, so I swapped it to the top. Not that it matters. The Vertex4 SSD has some firmware issues, which OCZ says they're aware of. But after disabling the option to put it to sleep the drives and disabling hybrid sleep, the SSD performs OK. I do get errors in the system logs about the SSD taking too long to wake from sleep. Which it does take a while. Hybrid sleep on the system causes a blue-screen / reboot almost every other time the system wakes, due to the SSD slow wake-up issue.

So far, I've encountered 3 "annoyances" with the HP system: 1> You can't mount anything in the spare drive bays, I had to go back and order two additional drive sleds at $40 each. 2> The second SATA header that consists of 4 channels at 3Gbps, is not bootable. Can only boot from the dual-port 6Gbps SATA header or the 8-port LSI SAS/SATA host. 3> The bottom X16 PCIe 3.0 slot would be perfect for double-width GPUs and other cards as the adjacent slot space at the very bottom of the system is a legacy PCI connector. Unfortunately, it's near impossible to wedge a large GPU card in there because of the amount of cables / connectors along the bottom of the motherboard there. Qualifies as a design flaw, IMO. On the other hand, if I decide to ditch the Quadro card (and I might if the GeForce performance is where I want it), then I'll go to the EVGA HydroCopper GPUs. I can then fit one in this lowest X16 slot and run the water lines out the slot space of the bottom legacy slot to an external radiator and reservoir. ...Or maybe that's too much of a pain. Not sure I want to get into external water systems and whatnot with this system. As of right now, it performs great with the Quadro 6000 card. Ill have to try other GPU combinations and see what the best way is for Resolve and other apps I'm running.

[Mark Orion]

I have a question I'd sure appreciate your opinion on. I have a 2008 Mac Pro 8 Core 2.8 kHz with 10GB of RAM.

Since I'm waiting to see when new Mac Pros arrive, would I get a noticeable performance improvement in CS6 if I upgrade to 32GB of RAM?

Thank you!

[Jeff Kilgroe]

Mark, upgrading to 32GB would help, but probably not in a huge way. If you're careful about not holding too many apps open at the same time, you should be able to squeak by just fine with what you have. That would be an expensive RAM upgrade and money probably better spent elsewhere. Do you intend to hold onto the '08 Mac for much longer?

[Jeff Kilgroe]

Just an FYI for those of you who have been following my posts regarding the dual Xeon E5 systems and HP Z820... The EVGA GTX580 Classified Ultra *WILL NOT* physically fit into the HP Z820. If I were to leave the PCI e bracket retainer loose and secure the cards with screws, then leave the PCIe airflow cover out of the mix, I can get the lid on the system. Not going to go that route.

Other than that, the GTX580 tests are looking good! GPU tests are progressing nicely and it looks like I may be sticking with dual GTX580 cards in my SuperMicro build and sticking with the Quadro 6000 in my Z820. I need a second card in the Z820 for additional CUDA acceleration. I'm torn between a lower clocked GTX580 (which will fit) or going with a Tesla C2075 which makes the HP into a "Maximus" configuration. Wish I had the Tesla here to benchmark with. I've heard it's pretty snappy, even though the specs on paper don't seem to show that.

Will have my final reports soon...

[Mark Orion]

Mark, upgrading to 32GB would help, but probably not in a huge way. If you're careful about not holding too many apps open at the same time, you should be able to squeak by just fine with what you have. That would be an expensive RAM upgrade and money probably better spent elsewhere. Do you intend to hold onto the '08 Mac for much longer?

Jeff, thanks so much for your insight. I'll probably hold on to the '08 Mac and wait to put my money into something that would provide a significant performance increase.

[Jose Lomeña]

Mark, I have the last 2x2.93 macpro. I bought it with 12gb, and when I updated to 48gb i saw an increase of 2x in speed with after effects render. Maybe it can be this specific ae project... maybe not. But 2x speed increase is a lot. With this specific project cpu only top at 500%. I think 10GB is too low for ae in some projects with 4k images. I think you need 12gb only for ae. I recommend to update at least to 16gb, maybe it didn't cost so much and after effects will have near double of free mem. (Maybe you can extract 2x1gb modules and add 2x4gb modules = 300$ with owc mem, i didn't test it)

[Jeff Kilgroe]

Mark, what is your current memory configuration now? The '08 Mac Pro has a dual-channel memory arrangement and juggling around memory modules or exchanging various modules could give you a significant speed increase. Going all the way to 32GB via 4GB modules of that kind doesn't make a whole lot of sense, IMO. It's going to run $950-$1100 to do it and the actual value of the '08 Mac Pro at this point is pretty low. Such is the way technological cookies crumble... Moving to 16GB via 8x2GB might be an OK compromise. You will get the fastest memory performance at least and generally speaking you shouldn't run out of RAM if you're conscious about which apps you have up and running. As for AE, it's a memory hog now that it's 64bit and if you're working with comps at 1080p and up with a lot of layers, you can eat up RAM pretty fast.

32GB is in a way a bit too much for the '08 Mac Pro or systems of that generation and prior. Sure, having it is great, but there are so many other performance restrictions and bottlenecks compared to newer hardware, it just seems questionable as to whether one should really dump more money into a 4 year old system.

Another option is to see if there's a used RAM kit out there you can pick up for cheap.

[Jason Enzer]

jeff,

do you have a dual z820? if so have you tested the memory bandwidth? i have been seeing less than desirable memory write benchmarks. in fact every dual processor c602 chipset board i have tested yields sub 11GB a sec memory write speed. even intel is looking into the issue as they are getting reports from numerous customers.

can you try maxxmem2 to assess if you havent already?

thx!

-jason


Just an FYI for those of you who have been following my posts regarding the dual Xeon E5 systems and HP Z820... The EVGA GTX580 Classified Ultra *WILL NOT* physically fit into the HP Z820. If I were to leave the PCI e bracket retainer loose and secure the cards with screws, then leave the PCIe airflow cover out of the mix, I can get the lid on the system. Not going to go that route.

Other than that, the GTX580 tests are looking good! GPU tests are progressing nicely and it looks like I may be sticking with dual GTX580 cards in my SuperMicro build and sticking with the Quadro 6000 in my Z820. I need a second card in the Z820 for additional CUDA acceleration. I'm torn between a lower clocked GTX580 (which will fit) or going with a Tesla C2075 which makes the HP into a "Maximus" configuration. Wish I had the Tesla here to benchmark with. I've heard it's pretty snappy, even though the specs on paper don't seem to show that.

Will have my final reports soon...

[Mark Orion]

Mark, what is your current memory configuration now? The '08 Mac Pro has a dual-channel memory arrangement and juggling around memory modules or exchanging various modules could give you a significant speed increase. Going all the way to 32GB via 4GB modules of that kind doesn't make a whole lot of sense, IMO. It's going to run $950-$1100 to do it and the actual value of the '08 Mac Pro at this point is pretty low. Such is the way technological cookies crumble... Moving to 16GB via 8x2GB might be an OK compromise. You will get the fastest memory performance at least and generally speaking you shouldn't run out of RAM if you're conscious about which apps you have up and running. As for AE, it's a memory hog now that it's 64bit and if you're working with comps at 1080p and up with a lot of layers, you can eat up RAM pretty fast.

32GB is in a way a bit too much for the '08 Mac Pro or systems of that generation and prior. Sure, having it is great, but there are so many other performance restrictions and bottlenecks compared to newer hardware, it just seems questionable as to whether one should really dump more money into a 4 year old system.

Another option is to see if there's a used RAM kit out there you can pick up for cheap.

Jeff, Here is the current configuration:

DIMM Riser B/DIMM 1 2GB
DIMM Riser B/DIMM 2 2GB
DIMM Riser A/DIMM 1 1GB
DIMM Riser A/DIMM 2 1GB
DIMM Riser B/DIMM 3 empty
DIMM Riser B/DIMM 4 empty
DIMM Riser A/DIMM 3 2GB
DIMM Riser A/DIMM 4 2GB

Looks like 4 more 2GB might be a good bang for the buck - thanks!

Thanks!

[Luke Boyce]

Jeff, man, I need your help. I just can't make up my mind what I want to do here. Do I go 2690's or 2687w's?

If I go 2687w, I'm thinking I'd have to go with 2 Corsair liquid cooling options. An H80 in the rear on 1 cpu and an H100 at the top of the case in a push/pull configuration.
OR do I go 2690's and not worry about the liquid cooling? The prices come out almost exactly the same. Not seeing anybody having tested the liquid cooling options with those cpu's it's hard to make a definitive decision and I don't know enough about this stuff to know any interesting differences in the cpu's. So what do you suggest? And if I go 2690, am I still going to need to purchase a premium air cooling option?

Decisions, decisions.

Also, I've been using 1 30" Apple Cinema Display for most of my career, but considered going with 2 Dell U2711's. Any thoughts? Will I hamper the 580's performance at all if I'm running two of those guys?

[Jeff Kilgroe]

jeff,

do you have a dual z820? if so have you tested the memory bandwidth? i have been seeing less than desirable memory write benchmarks. in fact every dual processor c602 chipset board i have tested yields sub 11GB a sec memory write speed. even intel is looking into the issue as they are getting reports from numerous customers.

can you try maxxmem2 to assess if you havent already?

thx!

-jason


Hi Jason,

I'm not really familiar with maxxmem2, but I gave it a try on the Z820 and attached the screenshot it saved. Not familiar with any of the tests it runs or how it runs them... some of the numbers seem lower than they should for what I would assume they're doing, but I'm not familiar with this benchmark app. I haven't really got into specific memory and CPU benchmarks just yet, but so far these systems seem crazy fast even compared to 12-core "Westmere" class Xeons running at 3 or 3.4 GHz.

[Jeff Kilgroe]

Jeff, man, I need your help. I just can't make up my mind what I want to do here. Do I go 2690's or 2687w's?

If I go 2687w, I'm thinking I'd have to go with 2 Corsair liquid cooling options. An H80 in the rear on 1 cpu and an H100 at the top of the case in a push/pull configuration.
OR do I go 2690's and not worry about the liquid cooling? The prices come out almost exactly the same. Not seeing anybody having tested the liquid cooling options with those cpu's it's hard to make a definitive decision and I don't know enough about this stuff to know any interesting differences in the cpu's. So what do you suggest? And if I go 2690, am I still going to need to purchase a premium air cooling option?

Decisions, decisions.

Also, I've been using 1 30" Apple Cinema Display for most of my career, but considered going with 2 Dell U2711's. Any thoughts? Will I hamper the 580's performance at all if I'm running two of those guys?

I'm not sure what to recommend for the liquid cooling. :( Based on everything that's out there now, I would go with TWO H100's if possible. Make sure the water block cooler portion covers the entire surface area of the CPU. I have not tried the Corsair ones myself, but they look to be the best option at the moment for what is actually available.

As for the GTX580, running two displays is not a problem at all. Of course, if you do run 3D or complex visuals that span both displays, you're pumping out that much more pixel and other data, so it does impact performance in that regard -- dual U2711's is the same as running a single 5120x1440 display. Anyway, I've been running dual 30" (mostly Dell 3007FPW) displays on my couple primary workstations, both PC and Mac for a few years now. Funny, the 27" actually feels small compared to the 30", even though they're the same horizontal resolution. I keep hoping that EIZO will figure out a way to drop the price on their 36" 4K monitor. I would gladly trade my dual 30's for one of those! :)

I've got dual GTX580 CLASSIFIED ULTRA cards running on my SuperMicro config. Seems like a great setup so far, other than being massively power-hungry and they'e a tight fit!!! IMO, the liquid cooled HydroCopper versions would have been the way to go, but I'm still running air cooling on this system. It seems to be working, even with the 2687W CPUs, but the box sounds like a jet at takeoff... It's going in a rack with a bunch of other noisemakers, so I don't really care about that. The HP system was nice and quiet, but now that I have it fully configured and up and running, it's not so quiet with the RED Rocket and Quadro 6000 whirring away inside. I still need a secondary GPU for it as the GTX580 Classified Ultra does not physically fit inside the HP system. I'm very interested in the soon to be released Tesla Kxx GPUs, but may just go with a lesser stock GTX580 that I know will fit. The 580 drives Resolve quite well.

[Eric Z]

Jeff & Luke,
Just for reference, here's a MaxxMem2 result of a very cheap PC with 4GB (2x 2GB) DDR2-800Mhz RAM sticks (ran on Windows XP Pro x86 SP3):
http://www3.picturepush.com/photo/a/8277626/640/8277626.png (http://picturepush.com/public/8277626)

[Luke Boyce]

I'm not sure what to recommend for the liquid cooling. :( Based on everything that's out there now, I would go with TWO H100's if possible. Make sure the water block cooler portion covers the entire surface area of the CPU. I have not tried the Corsair ones myself, but they look to be the best option at the moment for what is actually available. I'd like to go with 2 H100's, honestly, but I just can't figure out where I'd put that second one. It's basically 2 120mm fans and almost all the cases I've looked at only have room for 1 120mm in the rear. From what I can tell my best bet for air circulation is to have the push/pull configuration of the h100 at the top of the case and then a 120mm out the rear as the exhaust. That's why I was thinking the H80 for the 2nd CPU. It's basically the same as the H100, except the radiator is thicker in a 120mm size. The case I'm most interested in at the moment is the Corsair 800D, and just like most of the other full server cases out there, it only has the 1 x 120mm size in the rear as opposed to 2 of them that I would need for another H100. Or at least, that's what I can gather with my limited understanding.


As for the GTX580, running two displays is not a problem at all. Of course, if you do run 3D or complex visuals that span both displays, you're pumping out that much more pixel and other data, so it does impact performance in that regard -- dual U2711's is the same as running a single 5120x1440 display. Anyway, I've been running dual 30" (mostly Dell 3007FPW) displays on my couple primary workstations, both PC and Mac for a few years now. Funny, the 27" actually feels small compared to the 30", even though they're the same horizontal resolution. I keep hoping that EIZO will figure out a way to drop the price on their 36" 4K monitor. I would gladly trade my dual 30's for one of those! :) Ok, that's great to hear. We don't do any 3D work here, so I don't think that'd be an issue. I know what you mean about the 27" seeming smaller. I'm surrounded by 27" iMacs and I think my 30" Cinema Display almost seems to dwarf them sometimes, despite knowing full well that the horizontal resolution is the same. It seems that vertical resolution is more impactful than we like to give it credit for, being "children of the widescreen".


I've got dual GTX580 CLASSIFIED ULTRA cards running on my SuperMicro config. Seems like a great setup so far, other than being massively power-hungry and they'e a tight fit!!! IMO, the liquid cooled HydroCopper versions would have been the way to go, but I'm still running air cooling on this system. It seems to be working, even with the 2687W CPUs, but the box sounds like a jet at takeoff... It's going in a rack with a bunch of other noisemakers, so I don't really care about that. The HP system was nice and quiet, but now that I have it fully configured and up and running, it's not so quiet with the RED Rocket and Quadro 6000 whirring away inside. I still need a secondary GPU for it as the GTX580 Classified Ultra does not physically fit inside the HP system. I'm very interested in the soon to be released Tesla Kxx GPUs, but may just go with a lesser stock GTX580 that I know will fit. The 580 drives Resolve quite well.I will say, having used Mac Pro's all the time, it's going to take some getting used to with the fan noise. I'm actually considering getting some better quality fans to replace the stock ones that come with the Corsair Liquid Cooling units. I heard that they can be unnecessarily loud and I know it's going to bother me, more than likely. Any recommendations on quality, lower-noise fans? I've seen a few recommendations on other sites already, but thought I'd ask the man himself.

Thanks again!!

[Luke Boyce]

Make sure the water block cooler portion covers the entire surface area of the CPU. I have not tried the Corsair ones myself, but they look to be the best option at the moment for what is actually available.

FYI, found this video very quickly showing an H100 covering a LGA2011 CPU. Looks like it does cover the entire surface, from what I can tell. Skip to 3:17 to see the H100 going on.


http://youtu.be/H3myiMEW3EM?t=3m17s

[Jason Enzer]

jeff,

these numbers are in line with my findings. 6GB a sec on writes is abysmal... we are supposed to be seeing results that are significantly higher. when run in single processor modes the chipset is getting the speeds it should. only when running dual procs is the memory not performing as it should. its a HUGE problem for us. intel is aware and we are waiting for a response.

to get an idea how bad here is my laptops maxxmem2 results.

http://img84.imageshack.us/img84/3007/maxxmem2f4014.jpg


i know it says desktop but its a dell precision m6600 :)

15GB a sec on a year old laptop, and besting a 10k machine almost 3:1...

-jason


Hi Jason,

I'm not really familiar with maxxmem2, but I gave it a try on the Z820 and attached the screenshot it saved. Not familiar with any of the tests it runs or how it runs them... some of the numbers seem lower than they should for what I would assume they're doing, but I'm not familiar with this benchmark app. I haven't really got into specific memory and CPU benchmarks just yet, but so far these systems seem crazy fast even compared to 12-core "Westmere" class Xeons running at 3 or 3.4 GHz.

[Eric Z]

Jeff,
Looks like the new Tesla K20 will be out in November:
http://www.anandtech.com/show/5840/gtc-2012-part-1-nvidia-announces-gk104-based-tesla-k10-gk110-based-tesla-k20/2

http://www4.picturepush.com/photo/a/8282622/640/8282622.jpg (http://picturepush.com/public/8282622)

[Jeff Kilgroe]

Yeah, I'm trying to hold my breath for the K20. :) Will definitely be taking one for a spin as soon as I can get my hands on it. The Tesla K10 ships next week I'm told, but it's essentially a GTX690 with no video ports. I'm definitely curious to see how it scores with CS6 and Resolve, i'm really not sure what to expect. On one hand, it has 2GB more to work with than the current Quadro6000/ Tesla C2075 and is capable of a lot more GFLOPS... But it's limited to single-precision and I don't know how that fits into the environment of those two applications -- the primary ones I'll be using it for. I'm kinda thinking a good GTX580 is still going to be superior and a good interim solution until that K20 arrives. And by then, we may see other GeForce offerings. I find it interesting that nVidia has announced the Kepler-based Tesla cards, but no mention of Quadro. I've heard a rumor from two separate sources that thee will be no new Quadro cards on this generation of Kepler GPUs. Not sure what to make of that, but it's starting to look very credible.

[Subhadip Sen]

I have briefly tested GTX 670 2 GB with Premiere Pro CS6 and it is faster than GTX 580 3 GB once you start adding in a bunch of effects; similar if you are only editing. At least Premiere CS6, for now, seems to be largely single precision oriented. There's evidence that the same applies for Resolve, considering GTX 580 beats Quadro 6000 while the latter is well ahead on double precision.

Tesla K10 is actually GTX 690 underclocked, to keep the TDP under 225W (and more expensive, of course). It's great for power sensitive HPC environments where you add in a ton of Teslas, but for individual PCs GTX 690 is probably the best option, if you can find one. K10 does have 4 GB ECC RAM per GPU and there is no news of GTX 690 8 GB variants. At this time, GTX 690 looks largely like a PR stunt with next to no supply - I have read only 3 out of Nvidia's 18 partners were invited to sell 690, and all reference designs. Come to think of it, in 2 months GTX 680 has almost never been in stock! GTX 670 is pretty hard to find too despite being stockpiled of defective dies for 4 months. Nvidia are having critical issues manufacturing these GK104 chips.

The no-Kepler(GK104)-for-Quadro rumour is very credible. At best we will see an mid range Quadro 4000 replacement. We will probably have to wait till 2013 for a next-gen GK110 based Quadro.

[Nikhil Kamkolkar]

So I have to ask if you've considered buying a system like the HP Z820? I received one here today and there's a lot about it to like. There is another thread around here, where I compared my Z820 configuration to a similar build using the SuperMicro X9DAi motherboard.

Jeff, any chance you could point me to the thread that has your Z820 config? I'm going to go down that route for editing R3D w/out RED Rocket (atleast to start off with).

Many thanks in advance!
Nikhil

[Luke Boyce]

So I'm finally pulling the trigger on this system and getting ready to order the parts. One thing I was wondering is, I'll be running dual Dell U2410's from the GTX 580, but I need another GPU for a production monitor via HDMI. Was wondering if somebody can give me some advice on a simple GPU to accomplish this task.

[Patrick O'Sullivan]

Just resurrecting an old thread here.

About to pull the trigger on an HP Z820 and wanted to know what GPUs I should be looking at.

Any suggestions Jeff?

[Jeff Kilgroe]

I know it costs more, but I strongly recommend buying your graphics card from HP for this! They use customized firmware on all their video cards that helps to smooth operations with their workstations.

For now, the Quadro 6000 is the best option for the system for best all-around performance. Some of the higher-end gaming cards can beat it in some operations, but for overall performance, it's king. That said, I wouldn't recommend buying one right now. I would save some money and do the Quadro 4000 or do a GTX580 from EVGA or PNY for the time being. The reasoning behind that is the new Kepler based Quadros are to start shipping in the next few weeks and HP should have the K5000 and K6000 approved for the Z820 within the next 30 to 60 days.

I suppose your timing on the system purchase also plays a role. Depending on how soon you buy, those cards may be available as an option right from the get-go.


Intel will be launching updated Xeon CPUs early next year -- probably end of February. It is unclear as of yet if they will be drop-in compatible with the current Z820 workstation (and most other current Romley platform Xeon systems). They will use the same socket and same chipset, but will have different (lower) power requirements. I've found in the past that upgrading CPUs, especially in a dual-Xeon box like this, is usually not very economical. Often better to sell the workstation or assign it to another task and purchase a new one. Just letting you know that the new ones are coming around the corner fast this time rather than a 26 month wait. Happening for two reasons, the last iteration was delayed nearly a year behind schedule and the new iteration uses the 22nm fab process so Intel is eager to push forward at this point as quick as they can.

[Patrick O'Sullivan]

Thanks Jeff.

I am pulling the trigger on Monday so it looks like I'll have to settle on dual E5 2690s.

I've got a gtx 470 si I'll add a 580 and that should do me for Resolve.

Thanks again for the explanations.