Comments Locked

43 Comments

Back to Article

  • Blackbrrd - Wednesday, March 3, 2004 - link

    It would be nice to see how the Xeon 2,8GHz DP 533mhz bus (no L3 cache) does against the Opteron 242 set up, as the price for these are about the same. As it is the fastest Intel CPU I can buy from for instance DELL is a 3,2GHz DP with 1MB L3 cache.

    As it is, we will be buying an Opteron server quite soon, based on the outcome of these benchmarks.

    About using a ramdisk: what they have actually tested is how a java webserver performs on the different CPU's. I am working in a small company that uses a java webserver with quite a bit of business logic, not just db inserts updates and selects. This test was about as spot on as you can get.

    Happy to see a site that isn't only doing game pc benchmarking :)
  • Ben98SentraSE - Sunday, January 11, 2004 - link

    First off, GREAT article. I check this site several times a day to see when the next review(s) in this area come out! :)

    I am the IS Manager at a medium-sized credit union and we are converting our core processing application from one that runs on an overpriced Unisys A-Series mainframe (good riddance) to one that runs on Oracle and Windows Server 2003. From Anandtech's articles (as well as a few other reviews around the web) I plan on purchasing Opteron servers to run this new system. Today I can run 32-bit and be fast, and tomorrow when Oracle's AMD64 version of 9i comes out of developer release and either when the AMD64 version of Server 2003 comes out or we become comfortable enough with Linux+Oracle, we can be faster. The point of stating this is just another prop to Anandtech's team that their IT Computing reviews are affecting purchasing in the real world at places like where I work that can purchase the best equipment for the job and for the price, regardless of brand. (thankfully!)

    If I could ask for ANY changes to Anandtech.com's reviews, I'd request a larger focus to IT computing. It is hard for me to find sources of information on server-type performance benchmarks in as much depth as Anandtech goes into them.
  • Jason Clark - Saturday, December 20, 2003 - link

    There we go :) Less confusing zuni = jason clark.
  • Zuni - Saturday, December 20, 2003 - link

    Trog, sure I did Zuni = Jason Clark I'll change my nic to my real name shortly as it is confusing. I co-wrote the article with anand.

    Visual,trog:

    The database side of this equation is coming, we're just waiting on some cabling for our u320 drive chasis.
  • Visual - Saturday, December 20, 2003 - link

    Zuni, thanks for responding, whoever you are ;)

    Seems you'll need to run some DB benches on those cpus, or even DB and webserver at once, for this review to be complete... as for me, I'm curious about how much better the A64 can be in 64bit mode :)

    I'm eager to see your next articles, and I want to say thanks to the whole AnandTech team for the great site you're making :)
  • TrogdorJW - Friday, December 19, 2003 - link

    Ugh... I feel like we keep talking past each other. I know and understand that the database was not being used as part of *this* benchmark. My point is that database operations are often a pretty major part of many web servers out there, so, I am curious as to what performance is when you're running the database and web server and everything else together on one system. Consider it a request for a future benchmark, not a fault with the present case.

    As an example, say you have some business that doesn't have a whole lot of money to spend on a server, so they're looking at a Linux box running Apache web server and PHP/MySQL. Certainly, this setup on a single system will not be able to handle a huge amount of traffic, like the Anandtech.com web site. However, I would like to see how much traffic such a system *can* handle. And how does the Athlon 64/FX compare to the Pentium 4 in such an arena? Is disk I/O more of a factor, or does the Athlon architecture actually do better?

    Any chance of getting such a benchmark done? Not necessarily with those specific applications, of course, but something similar? Or, if that's a pointless benchmark, please give reasons... maybe it's already been tested by others? Whatever.

    Thanks for the responses, though. And you never answered: is "Zuni" Anand or Jason, or is it someone else who just worked with those two on the article?
  • Zuni - Friday, December 19, 2003 - link

    Essentially using a ramdisk allowed neither cpu to have a limit on its scalability. That's what we were after, and doing that without having expensive raid arrays that are only used when we run a server test. It provided no benefit to either manufacturer it just allowed neither to be limited in any fashion. It worked well, and the numbers show that. I sure hope we can run itanium, we're trying to get ahold of one for you guys.

    Cheers, and everyone have a great holiday!.
  • Falco. - Friday, December 19, 2003 - link

    Zuni, thanks for all the replies, and please keep in mind that most of the replies are not from me, but from a friend of mine that knows ALOT more about this then i do :-)
  • Abraxas - Friday, December 19, 2003 - link

    trogdorjw, in the article it states that the database WAS a bottleneck as you've pointed out, however, it does say "particularly with the opteron." to me this is saying that the database was not a bottleneck with the xeon servers and moreso with the opteron, ie the opteron was only maxed out when using the ramdisk and did not have as much of a performance advantage without it. this would not mean that the opteron had NO advantage over intel with a more normal disk setup. the article also pointed out that even the lower-end opterons (240) would have an advantage over the fastest xeons. this would be even more true under your suggested "real world" setting.

    i can't wait to see the itanic vs opteron comparison :)
  • Zuni - Friday, December 19, 2003 - link

    Thanks for the feedback, appreciate it. If you have further suggestions or applications you feel could be tested post them here.

    Cheers
  • Zuni - Friday, December 19, 2003 - link

    "That entirely depends on the application being tested, and the load on the server, and the memory on the server. If you're calling the same 10 files over and over again then they'll likely be cached. If you have a true application where someone hits a page and creates a 600 megabyte recordset"

    Sure I/O usage would depend on the type of web application being tested. However, most web applications feeding off a database backend would not pull 600 mb recordsets :) Pulling such record sets would be a fairly silly thing to do in most any circumstance. Our test was absolutely not hitting I/O on the webservers, the template cache is more than enough to hold the templates being tested. In fact as I'm writing this I pulled the disk time on one of Anands busiest web servers and its showing less than 1% of disk time :) If an application is well written, and the webserver has a decent amount of memory (most do) then I/O should not be a factor in a run of the mill application such as FuseTalk.

    "This smacks of rhetoric. ASP and ASP.NET are also 'very good reference points' as enterprise level application environments. "

    Not at all, although I can see how you could draw that conclusion. FuseTalk is currently being ported to .NET, once we have a stable working version or another app we deem worthy of a real world test we’d be glad to run tests on .NET as well. I quite like the platform. This is just the beginning of our server testing and developing real world tests. It takes time, and a lot of it to develop and run theses tests.

    Cheers
  • Zuni - Friday, December 19, 2003 - link

    Trog, again RAMDISK was used so that the application server was free to tax the cpus as much as possible instead of waiting for data requests from the database. This has nothing to do with normal or abnormal. We aren’t testing the database here we’re testing the application server, and we wanted no bottlenecks on the backend to cause any skewing on the front end. We don’t have fibre channel backend database hardware just laying around, thus a RAMDISK was used to remove the I/O bottleneck.

    Again, the database is not in question here or being tested so its irrelevant.

    Cheers.
  • Falco. - Friday, December 19, 2003 - link

    " 1) disk subsystem is irrelevant in a web application driven by a database backend. Most any web application server caches the templates in memory so I/O is not a factor. However, each web machine had a 36gb u320 scsi drive. "

    That entirely depends on the application being tested, and the load on the server, and the memory on the server. If you're calling the same 10 files over and over again then they'll likely be cached. If you have a true application where someone hits a page and creates a 600 megabyte recordset, your disk IO is gonna come in handy right quick. Saying "I/O is not a factor" is misleading.

    However, I'm glad they responed the way they did, because if you hvae a testbed that starts to swap or spin out to disk, that doesn't test the CPU as much. I was expecting they'd come back and say they used a test set that was cached compiled templates (which is a good thing). My question was aimed at eliminating it as a possible variable.


    " 2) Not sure I understand your friends point really... Of course ColdFusion tests performance, we used FuseTalk as the web application and ColdFusion as the web application server. "

    That's not entirely what I meant. Cold Fusion is a middleware layer. It's not a "web server" as per the title of the review (Cold Fusion actually can include its own webserver if you want it to).

    I found the title of the review to be somewhat misleading, since it implied they were testing a web server and not an application server.


    " ColdFusion in its current version is a J2EE server which allows you to write in the ColdFusion language. After its compiled its the same compiled byte code as any J2EE app. "

    Correct; it's an interpreter (CF) running on an interpreted language (Java). It's a good test in that it has multiple subsystems communicating together on top of having to handle actual web serving such as setting up sessions and opening and closing connections.



    " We may include .NET in the future, but J2EE is a very good reference point as its a enterprise level application environment. "

    This smacks of rhetoric. ASP and ASP.NET are also 'very good reference points' as enterprise level application environments. A whole huge great pile of systems run ASP. It struck me that it would have been worthwhile to round out the review by testing a number of common web application server configurations:

    * Pure J2EE
    * ColdFusion
    * ASP
    * ASP.NET
    * PHP (although this would not test IIS, as most PHP systems drive it with Apache -- mind you, a test with Apache would be equally interesting)


    " Sustained load throughout the test keeping the CPU working in the 80-100% bracket. We measure performance by how long the cpu took to run each code template from start to end. (this was all detailed in the review btw.) "

    This part I understood and took for granted. Interestingly if they had sustained 80-100% CPU load, this is an added shine for the AMD parts, because past 70% load on your webserver you should be adding another box or upgrading your CPUs.
  • TrogdorJW - Thursday, December 18, 2003 - link

    Hey Zuni... who are you? Anand? Or "just" an Anandtech employee (or reviewer or whatever)? Anyway, let me clarify:

    "The entire reason we used a RAMDISK was to ISOLATE the cpu not to hold it back by something else."

    Exactly. Ergo, without the RAMDISK holding the database, file I/O can be a much bigger factor than CPU speed. Yes, you can get bigger and faster disk subsystems, but that's not always the way it's done. Anyway, it *sounds* to me like a more "normal" setup on the database side would end up being a major bottleneck on the overall throughput, so that either of these systems is overkill. You need a DB system with a very high I/O rate first, and then looking at these faster CPUs comes second.

    On a separate thought, are there going to be some benchmarks *without* using a separate DB server (which used a RAMDISK)? I understand that your site may not run that way, but the vast majority of web servers do, I believe. If you have one server running both the database backend as well as the application services, what do the benchmarks look like? That would be something to look at, surely?
  • Falco. - Thursday, December 18, 2003 - link

    thanks Zuni, i'll let him know and post back any replies he may have
  • Zuni - Thursday, December 18, 2003 - link

    Falco,

    1) disk subsystem is irrelevant in a web application driven by a database backend. Most any web application server caches the templates in memory so I/O is not a factor. However, each web machine had a 36gb u320 scsi drive.

    2) Not sure I understand your friends point really... Of course ColdFusion tests performance, we used FuseTalk as the web application and ColdFusion as the web application server. ColdFusion in its current version is a J2EE server which allows you to write in the ColdFusion language. After its compiled its the same compiled byte code as any J2EE app. We may include .NET in the future, but J2EE is a very good reference point as its a enterprise level application environment.

    3) Sustained load throughout the test keeping the CPU working in the 80-100% bracket. We measure performance by how long the cpu took to run each code template from start to end. (this was all detailed in the review btw.)
  • Falco. - Thursday, December 18, 2003 - link

    a friend has some questions about this review :

    Did they get into details about the disk subsystems on each of the web servers? I know they tried to keep the database in ramdisk (even though SQL server will do that anyway if the database is small enough).

    What sort of web application were they running? I find it curious they were using a Coldfusion application as well, since that doesn't test web response as much as the cold fusion runtime services; they should have used an ASP or ASP.NET application test, at least as an additional data point.

    What did they do to test web response time? Sustained load? stressed load? burst load?

    thats what hes asking in a forum i frequent...
  • Icewind - Thursday, December 18, 2003 - link

    Man, how the hell do you learn and keep up with all this server technology, both hardware and software? Im having a hardtime just keeping desktop technology and changes....
  • Zuni - Thursday, December 18, 2003 - link

    skiboysteve, because thats what was sent with the systems :) 8GB of it.
  • Zuni - Thursday, December 18, 2003 - link

    Visual, the db server didnt need hardly any ram the database was barely 50 MB :) Again the point of the article was to eliminate the db as anything but a boundless data resorce so the web applications servers could process pages as fast as possible, keeping the cpu taxed. This was a 32 bit test, OS etc. We're trying to get ahold of a 64 JVM and we're looking at some UNIX based 64 bit tests for another article.

    Cheers!
  • Zuni - Thursday, December 18, 2003 - link

    TrogdorJW, I think you missed the point behind why a ramdisk was used. Every web application is governed by the time in which it can retrieve information from the database. We are testing the performance of a cpu in a web application server environment not in a database environment. The entire reason we used a RAMDISK was to ISOLATE the cpu not to hold it back by something else. Obviously in a production environment you hardly ever use a ramdisk. Most of them only format up to 4GB using AWE extensions. Again we wanted to ensure that the cpus were being taxed and were not being bottlenecked by the database in this instance.

    Cheers, and thanks for the feedback!
  • Visual - Thursday, December 18, 2003 - link

    I got a couple of questions...

    1) Is this Microsoft Windows 2003 Enterprise Server that ran on the opteron a 64-bit OS version? And is the IIS a 64-bit app? I guess not... But I didnt see it mentioned in the article, nor did I see a mention of the possible performance increase for the opterons with 64-bit software. I skipped a big part of the article though, because I'm in a hurry now, so if that info is in the article, forgive me.

    2) Why does the DB server has just 2GB ram? and probably 1GB used up for the ramdrive, so just 1GB RAM left for the DB server? While the webservers had a whooping 4GB ram... I dont have a clue about servers, but I'd imagine the database server would need more ram than the web server, seeing that it is the one dealing with all the data. Well I guess if the whole database could fit on a ramdrive of 1GB, it was small enough so RAM size would not be an issue...
  • lockup - Thursday, December 18, 2003 - link

    TrogdorJW, although the database had to use a ramdisk to prevent it from being the bottleneck in this case, that doesn't mean these benchmarks are less valid in the real world.

    There are some pretty hefty databases out there with wintel appservers on the front end. Also, there are web applications that are doing a bit more than taking an id, getting some data on a select statement and formatting it in HTML. For systems where performance has been a focus, you may see methods implemented in the application to take processing away from the db and in to the application.

  • skiboysteve - Thursday, December 18, 2003 - link

    you talked about why the new revison k8s can use ddr400, then you use ddr333

    why
  • TrogdorJW - Thursday, December 18, 2003 - link

    Wow... nice performance from the AMD system!

    Too bad that, for the time being, it won't help a whole lot. The same people who bought Athlon MP will look to the Opteron as an upgrade, but unfortunately, most of the Intel camps will probably remain Intel for the present. Things to think about:

    1) As this review so clearly shows, servers are full of different bottlenecks. When tied to a DB server with a RamDisk holding the entire database, the Opteron killed the Xeon. But, it also mentions how in order to put any real strain on either CPU, they *had* to use a RamDisk on the DB server. That's not an option with real databases, usually... unless you're into having RamDisks that are several GBs in size! So, in real world scenarios, it looks as though the conclusion is that you'll need to have one hell of a database server before you even think about going with a server using either of these CPUs.

    2) AMD is not "easy as Dell." I hate Dell. We use them at my work. My company is a Fortune 500 company (top 100, actually). Guess who isn't going to use AMD in the office anytime soon....

    3) This sort of goes off of the last point, but we have 8 Dell servers in the data center I work in. All of them are Xeon dual proc systems. Actual amount of work done by these systems? Virtually NOTHING! Two of the systems aren't even in use at all! So, while the Opteron may be much more powerful, for big companies that may not even matter. We could have Pentium III Xeon servers, and they would still be more than fast enough. (We also have to IBM R6000 servers running a custom client-server application. Neither Intel nor AMD will get this server's spot!)

    4) CRAMITPAL supports AMD. People like CRAMITPAL support AMD. That does much more harm than good! I don't know about you, but if I had some stupid ass moron like CRAM trying to convince me to use an AMD system, I would do anything but what he recommended. That's where fanatacism gets you. So next time any of you think of bashing Intel and acting the fanboy, remember that emotions only hurt in big business. You can show cold hard facts, but if you're trying to get a CEO to switch from Intel to AMD, you're going to need a whole helluvalot of support. As the saying goes, no one ever got fired for recommending Intel. Sad, but true.

    5) The Itanium stuff... well, it will certainly be interesting to see a matchup. 2 and 4 way Itanium servers are not the real problem, though. Sure, the Opteron might win out in those competitions (I'll wait to see the numbers), but the real Itanium stronghold is stuff like the 128 way Itanium systems that are mentioned. That's just way out of the league of AMD right now. Sorry. But I hope that in the 64-bit competition, we'll get to see some systems with 8 GB of RAM or more in them, since that's where 64 bit computing actually becomes a real necessity.

    6) Also concerning the Itanium, SpecINT and SpecFP numbers can be almost completely meaningless as a performance comparison, depending on what you're using the server for. I don't have any in depth knowledge of the inner workings of Itanium servers, motherboards, etc. However, if I/O becomes a major part of the benchmarking, I believe I've heard that the Itanium does quite well in that arena. Like I said before, we'll have to actually see the benches before anything can be determined. (Unless someone has a *reputable* link to a comparison between Itanium 2 and Opteron?)

    Okay, enough babble. Cool article. Cool system. I don't need it - never will, probably - but cool, regardless.
  • sprockkets - Wednesday, December 17, 2003 - link

    I love glueless operation. But that's just it, we won't sell AMD as #1 even though it's better cause of Intel's crap as being better somehow. Yeah, they are much slower, but all they care about is whether Dell sells them. At least there are some who don't listen to what's the standard, i.e. we must use microsoft instead of linux and intel instead of AMD.
  • TheRealMandak - Wednesday, December 17, 2003 - link

    How come you did not use PC3200 DIMM's ?

    I look forward to the next test.

    If any want more of this try: http://www.aceshardware.com/read.jsp?id=60000275
  • Zuni - Wednesday, December 17, 2003 - link

    Ryan, the RAMDISK actually has nothing to do with the performance of the web servers..Remember we used a separate db server for the backend. The webservers didnt run a RAMDISK the databse server did..We measured the performance of the webservers not the database server.
  • RyanVM - Wednesday, December 17, 2003 - link

    I'm wondering how much the memory controller affects RAMDISK performance..

    It seems to me that a RAMDISK would be eating up the Xeon's already limited amount of available bandwidth. Add to that the on-die controller of the Opteron, and suddenly a RAMDISK might not be as apples-to-apples as one would hope.

    I'd be interested in seeing performance testing with a lot of fast drives in a RAID5 array :P.
  • Pandaren - Wednesday, December 17, 2003 - link

    Superbike, don't encourage this extremist. Cult tactics promoted by Apple dihards never worked (2% marketshare), and the same tactics won't help AMD.

    Argue on facts, not emotional fluff. Turning this into a "Good versus Evil" argument only destroys the credibility of the person posting the argument (and hurts the reputation of AnandTech in general if people see nothing but flame wars).
  • Superbike - Wednesday, December 17, 2003 - link

    CRAMITPAL right as always!
  • Jeff7181 - Wednesday, December 17, 2003 - link

    You'd think some people here have a huge investment in AMD the way they touch their balls every time AMD comes out ahead in a benchmark.

    Anyway, it's nice to see some benchmarks that clearly show what AMD processors are capable of... only other thing I'd like to see is the cost of the configurations used. That would even extend AMD's "lead."
  • morcegovermelho - Wednesday, December 17, 2003 - link

    Ooops...
    The last sentence should be read as:
    try in calculator 141 + 82.3%. The result is 257,043.
  • morcegovermelho - Wednesday, December 17, 2003 - link

    quote:
    "The Opteron 248 setup managed to outperform Intel’s fastest, largest cache Xeon MP by a whopping 45%"
    I think the number should be 82,3%.
    If the Opteron was twice as fast (100% faster) as the Xeon the Average Request Time would be half of 257ms (128.5ms). The Opteron Average Request Time is 141ms (82% faster than Xeon).
    Try in calculator: 141 + 82%. The result is 257,043.
  • Shinei - Wednesday, December 17, 2003 - link

    The message is clear: Opteron wins, flawless victory. Now if only I could AFFORD a 248 setup... ;)
  • RZaakir - Wednesday, December 17, 2003 - link

    "it would of been nice to have taken out a singnal(sic) opteron also so(sic) see 1x proformance."

    Knowing how well Opteron chips scale, this was probably a decision made out of mercy for Intel.
  • Nehemoth - Wednesday, December 17, 2003 - link

    Awesome
  • dvinnen - Wednesday, December 17, 2003 - link

    it would of been nice to have taken out a singnal opteron also so see 1x proformance.
  • jerkweed - Wednesday, December 17, 2003 - link

    Quote: Intel was not very receptive to the idea of doing a head-to-head; not out of a fear of losing, but out of a desire not to lend AMD any credibility by showing that the Opteron is indeed a competitor to the Itanium 2.

    That might be what Intel told AT, but honestly, Intel is terrified of seeing a head-to-head benchmark for an application like this. Itanium/Itanium 2 (known by most HPC/64-bit gearheads as 'Itanic') will show numbers much slower than even their Xeons for a web benchmark. The vast majority of all web-server cpu usage is INT specific... look at the numbers for spec INT yourself:
    http://www.spec.org/cpu2000/results/res2003q4/
  • Falco. - Wednesday, December 17, 2003 - link

    all i can say is damn...
    can't wait for that 4 way shootout and the opteron vs itanium test ...

  • Pandaren - Wednesday, December 17, 2003 - link

    cramitpal, speaking as an advocate of the k8 architecture, I have to say that you are acting like an asshat. cut the holy jihad crap and jerry falwell flaming - frankly you sound like a Steve Jobs worshiper or a Linux-happy script kiddie.

    k8 is a nice architecture for servers. it has been clear to me for some time that intel designed netburst for multimedia applications, and this emphasis has hurt netburst Xeons ever since the days of the Willamette (anyone remember the Willamette based Xeons getting matched or beaten by Pentium III based Xeons?)
  • PaperclipGod - Wednesday, December 17, 2003 - link

    I really enjoyed this article. Very well written. Looking forward to the Itanium 2 comparison.
  • CRAMITPAL - Wednesday, December 17, 2003 - link

    What a humbling experience for Intel... Results mirror other website tests of the latest and greatest Xeon w/L3 cache. AMD just HAMMERS Intel's Xeon into the ground.

    You would think Intel would be anxious to provide a 2P Itanic for comparison, wouldn't you??? Do you think Intel is afraid enterprise will realize that Opteron can provide Itanic 64-bit performance, and superior 32-bit performance for tens of thousands less??? The clowns in Satan Clara must STILL think the World is full of sheep! This review should make the Intel fanboys go POSTAL again.

    SOS same dumbass Intel fanboys. Maybe these confused fanboys are actually Intel SpinMeisters looking to keep their jobs as Intel's sales and market share diminish???

Log in

Don't have an account? Sign up now