How fast is PearPC: Benchmarks

[Ribtorus]

My experience with PearPC gives the following rough estimates of the slowdowns compared to the host that I have seen so far:

Photoshop 7 : [b:8c2a3b92c2]16.5X[/b:8c2a3b92c2] - considering an array of different filtres and actions.
Bryce5 : [b:8c2a3b92c2]19.2X[/b:8c2a3b92c2]
CineBench 2003 : [b:8c2a3b92c2]50X[/b:8c2a3b92c2]
Strata3D CX : [b:8c2a3b92c2]50X[/b:8c2a3b92c2]
Stuffit Expander : [b:8c2a3b92c2]2.5X[/b:8c2a3b92c2]
Quicktime : [b:8c2a3b92c2]9X[/b:8c2a3b92c2] - export .MOV to .AVI
Quicktime : [b:8c2a3b92c2]50X[/b:8c2a3b92c2] - convert .aif or .wav to .OGG using plugin

Each of those apps have exact version equivalents in Windows.

[Ribtorus]

Another small quicktime comparison.

This time I imported a .MOV and exported to .AVI, but at all the highest settings. I saw a [b:107d8a8113]10X[/b:107d8a8113] slowdown over the host machine, which is in pretty good agreement with the other Quiktime to AVI test I did.

Then I also exported the .MOV to a streaming DV file. This type of conversion must take a different route through the CPU, because the slowdown over the host machine went back up to almost [b:107d8a8113]16X[/b:107d8a8113].

That [b:107d8a8113]16X[/b:107d8a8113] slowdown factor sure seems to show up a lot in the CPU intensive tests I have run.

[Ribtorus]

Given the recent buzz about CherryOS, I'd like to see at least an XBench score and a PS7Bench trial with the 50MB file as well as Cinebench.

My gut feeling tells me the 80% host speed claimed is limited to certain tasks and does not represent the sort of performance one could expect with CPU intensive applications.

But I hope I'm proven wrong.

[Ribtorus]

Another benchmark. This time I used a complex action script in Photoshop CS working on a 75MB file. Instead of comparing the time to complete to the host PC, or even to the reported CPU speed given by the system profiler, I compared the speed between two versions of PearPC.

The first run was done on the G3 Athlon optimized PearPC from:

http://www.richardgoodwin.com/pearpc/cvsbuilds.php

The second run was done using the experimental G4 altivec version.

Both versions claim my emulated mac is 1,06 GHz with 896MB ram. The difference being one is reported as a G3 and the other as a G4.

Time to complete the action script with the G3 version: 14 minutes 30 seconds.

Time to complete the action script with the Altivec version: 21 minutes 10 seconds.

This is not the only instance where I have noticed generally slower operation with Altivec builds.

[Ribtorus]

Continuing the comparison of a G4 Altivec build vs a G3 build, I ran Pi Calculator and Speedrun.

Both benchmarks showed no appreciable difference whatsoever.

Pi Calculator turned in a time of [b:6b9e833134]76[/b:6b9e833134] seconds with the G4 build, and [b:6b9e833134]74[/b:6b9e833134] seconds with the G3.

SpeedRun turned in an overall score of [b:6b9e833134]85[/b:6b9e833134] in both tests.

[mmischke]

(Reposted here at trygvebw's request)

I've got an 800 MHz G3 iBook (awesome machine!), and I've been wanting to compare PearPC performance (on my 2.6 GHz P4 PC) to it for awhile now. I just got around to it. Please see the link below for additional details.

I used Xbench 1.1.3 and Panther (all current patches) for all tests.

My measurements are decidedly unscientific. I did one run per configuration, rather than the 3 runs, plus standard deviation, that might constitute valid scientific data. Most tests are pretty reproduceable, but disk tests tend to sway a good 20%. Large files on XP are very susceptible to fragmentation. The XP defragger (and Diskeeper, its big brother) don't seem to be able to do much with large files. Check out PerfectDisk from [url=http://]http://www.raxco.com/[/url:121020e1e5]. I've been pretty happy with its handling of large files.

In summary, my iBook smokes PearPC in CPU and Open GL tests, but gets beaten in memory access. Despite the fact that my trusty G3 is beginning to show its age, I'm impressed that PearPC can give it a run for its money in a couple of areas.

For PearPC, Altivec seems to have an edge over standard in both CPU and memory access. I was surprised that the SDL Altivec build didn't ace out the non-SDL Altivec build in the Quartz and Open GL tests, given that I have a decent graphics card in my PC (I thougth SDL used DirectX).

I leave it to you to take my results and subject them to additional analyses. If you do, please share your results with us. Either way, a glance at my findings suggests that PearPC is no slouch, given the overhead is incurs (yeah, right, CherryOS...). Granted, my iBook is no speed demon, but I use it day in and day out.

[url]http://www.mischke.org/xbench.htm[/url:121020e1e5].

Mark[/url]

[Ribtorus]

Using mmischke's example, I ran Xbench with the latest 0.4pre Athlon build from richardgoodwin.com. I ran a G3 and a G4 version and then ran the G3 0.31 PearPC.

[b:0ce4e45012]0.31 G3 SDL JIT[/b:0ce4e45012]

CPU: 0.36
Thread: 27.20
Memory: 43.71
Quartz: 23.44
OpenGL: 2.12
Disk: 44.71

[b:0ce4e45012]0.4Pre AthlonXP[/b:0ce4e45012]

CPU: 0.58
Thread: 27.75
Memory: 48.70
Quartz: 24.52
OpenGL: 2.16
U/I: 26.81
Disk: 47.96

[b:0ce4e45012]0.4pre Altivec[/b:0ce4e45012]

CPU: 1.55
Thread: 25.63
memory: 57.81
Quartz: 24.30
OpenGL: 2.43
U/I: 25.99
Disk: 48.75

I then ran Photoshop CS in each version.

I used a complex action script to create and manipulate a 75MB file with a variety of functions. I recorded the time to complete the script.

[b:0ce4e45012]0.31 SDL JIT[/b:0ce4e45012], 14 minutes 5 seconds

[b:0ce4e45012]0.4pre Altivec,[/b:0ce4e45012] failed to complete (Photoshop just stalled out 2/3 through the script)

[b:0ce4e45012]0.4pre AthlonXP[/b:0ce4e45012], 14 minutes 30 seconds

I can see no obvious relationship between XBench's CPU score and actual CPU intensive processes, like Photoshop. For reference, the host completes the same script in about 45 seconds.

I ran the SpeedRun benchmark as well.

[b:0ce4e45012]0.31 SDL JIT[/b:0ce4e45012]

Graphics: 18
Hard Drive: 119
Processor: 58
ram: 142
Overall: 82

[b:0ce4e45012]0.4pre AthlonXP[/b:0ce4e45012]

Graphics: 19
hard Drive: 119
Processor: 56
Ram: 145
Overall: 85

[b:0ce4e45012]0.4pre Altivec[/b:0ce4e45012]

Graphics: 20
Hard Drive: 123
Processor: 58
Ram: 147
Overall: 87

According to SpeedRun's report, there was no significant improvement in any run to alter my position in the results chart that Speedrun displays.

So far, I have not been able to relate XBench's CPU score to the benchmarked performance in actual CPU intensive applications.

[Ribtorus]

If anyone has Photoshop 7 or Photoshop CS and would like to run either the PS7bench script, or the Photoshop CS script, I have provided links here. Photoshop action scripts are small files. I believe Photoshop CS is available as a demo download from Adobe. It just requires a free registration.

I don't know if Bryce5 is available as a demo, but Strata 3D CX certainly is, and I can link to the site where I got the test render file and the instructions as well, if anyone wants to try.


I'll post what links I can here:

http://www.strata.com/

http://www.stratacafe.com/benchmark_form.asp
It's very important to follow the instructions carefully, or the times will be meaningless.

http://www.cinebench.com/
Cinebench is another self contained easy to run benchmark.

http://homepage.mac.com/alk/picalculator.html
Note the three versions available at the bottom of the page. You can even benchmark basilisk II, since there is a 68k version.

http://mac.megalink.com/files/speedrun.sit

SpeedRun is a very quick and easy benchmark.

http://www.xbench.com/

Easy to run, but takes a while. The online browser shows your results in relation to actual macs. I have reservations about how meaningful XBench is.

For Adobe Photoshop CS, I think it's available from Adobe as a demo.

here's a link to the CS action script I used for Photoshop CS.

http://www.s93081652.onlinehome.us./Sha ... nch_v1.atn

Unfortunately Photoshop 7 is no longer available for demo download, but if you have it, here's the link to the PS7bench script;

http://www.geocities.com/Paris/Cafe/436 ... nchAdv.zip

When running either of the photoshop scripts, it's critical that you set the Photoshop preferences history state to "1" and give Photoshop as much ram as you can spare. Also, for the Photoshop 7 script, be sure you have the "timing" indicator set at the bottom of the Photoshop window, and not file size or any of the other options. This is the indicator from which you record the times for each action.

For Photoshop 7 and CS, don't worry if the image just appears black, it's a real image and it appears fine when opened in other imaging apps. Photoshop CS benchmark requires a stopwatch or watch with a second hand.

Of course, Strata3D, Cinebench, and Photoshop are also available for Windows, so you can directly compare your PearPC slowdown factor against your host Windows machine.

I have not attempted any PearPC benchmarks with a Linux host.

[Ribtorus]

Here's a small comparison of running a 0.4pre build (G3) with Windows XP and bartPE, which for the purposes here can be thought of as a [b:b7c90dcef7]very[/b:b7c90dcef7] stripped down version of WindowsXP. In both cases, ram was set to 512MB with the display at 1024x768x15.

Xbench windowsXP
CPU: 0.[b:b7c90dcef7]57[/b:b7c90dcef7]
Thread: [b:b7c90dcef7]27.51[/b:b7c90dcef7]
Memory: [b:b7c90dcef7]48.29[/b:b7c90dcef7]
Quartz: [b:b7c90dcef7]24.52[/b:b7c90dcef7]
OpenGL: [b:b7c90dcef7]2.15[/b:b7c90dcef7]
U/I: [b:b7c90dcef7]26.10[/b:b7c90dcef7]
Disk: [b:b7c90dcef7]47.82[/b:b7c90dcef7]

Xbench BartPE
CPU: [b:b7c90dcef7]0.55[/b:b7c90dcef7]
Thread: [b:b7c90dcef7]26.59[/b:b7c90dcef7]
Memory: [b:b7c90dcef7]48.43[/b:b7c90dcef7]
Quartz: [b:b7c90dcef7]14.91[/b:b7c90dcef7]
OpenGL: [b:b7c90dcef7]2.11[/b:b7c90dcef7]
U/I: [b:b7c90dcef7]20.33[/b:b7c90dcef7]
Disk: [b:b7c90dcef7]52.28[/b:b7c90dcef7]

Not much change, except for Quartz and U/I (User Interface), here we see a big drop with bartPE. We see a small gain with hard disk performance.

SpeedRun WindowsXP
Graphics: [b:b7c90dcef7]19[/b:b7c90dcef7]
H/D : [b:b7c90dcef7]118[/b:b7c90dcef7]
Processor: [b:b7c90dcef7]56[/b:b7c90dcef7]
Ram: [b:b7c90dcef7]144[/b:b7c90dcef7]
Overall: [b:b7c90dcef7]85[/b:b7c90dcef7]

SpeedRun BartPE
Graphics: [b:b7c90dcef7]8[/b:b7c90dcef7]
H/D : [b:b7c90dcef7]97[/b:b7c90dcef7]
Processor: [b:b7c90dcef7]39[/b:b7c90dcef7]
Ram: [b:b7c90dcef7]63[/b:b7c90dcef7]
Overall: [b:b7c90dcef7]52[/b:b7c90dcef7]

SpeedRun shows a marked drop in performance using BartPE.

Pi Calculator WindowsXP: [b:b7c90dcef7]77[/b:b7c90dcef7] seconds

Pi calculator BartPE: [b:b7c90dcef7]132[/b:b7c90dcef7] seconds

Pi Calculator shows a marked drop in performance as well. This is related to the drop in video performance because of the video-dependent settings selected in Pi calculator.

PhotoshopCS Windows XP: [b:b7c90dcef7]15 minutes 34 seconds[/b:b7c90dcef7]

PhotoshopCS bartPE: [b:b7c90dcef7]15 minutes 45 seconds[/b:b7c90dcef7]

Photoshop CS shows no significant difference.

So far, I can see no advantage to using BartPE in its default form as a substitute for Windows as far as PearPC performance goes.

This is consistent with my earlier benchmark where I booted XP in safe mode with minimal services running: no advantage as far as PearPC is concerned.

I imagine it's because what's slowing PearPC down has far less to do with system overhead and more to do with registers and other fundamental architectural differences between the x86 and the PowerPC.

Perhaps efforts at running PearPC with a minimal Linux boot kernel, [i:b7c90dcef7]with the aim of improving performance[/i:b7c90dcef7], are a bit misplaced at this stage. However, I can see where less powerful host machines might gain a small benefit. But these older, slower machines aren't really good candidates for running emulations anyway. Even native G3's can struggle with OS X, let alone an emulated G3 on a 1.5GHz x86.