The most critical component is the PC operator actually :devil:
With that said there is no single critical physical PC component, although there are several critical components (plural).
1) Monitor - In conjunction with the graphics card this is the part of the most critical component chain for good graphics display. A quality CRT is required with high levels of contrast, small onscreen pixel size and a flat low reflectivity screen. It is also highly desirable to have individual RGB control and adjustable white point temperatures. LCD monitors don't work well due to low contrast ratios and limited bit depth (16 bit typical).
CRT monitors need to be replaced every few years as they fade and their illumination levels drop.
2) Graphics card. 3D performance is not critical, what is critical is 2D colour accuracy and having a high refresh rate and a 32 bit or higher bit depth per pixel.
3) RAM - The more the merrier if working with PhotoShop. 2 Gig is a good starting point and if you are using a 3rd party RAW/NEF converter such as Nikon Capture or Bibble add an extra Gig for it and at least 512mb for the OS. This means 4 Gigs of RAM is a good starting point.
4) FAST Hard Disk for working space. - There's nothing worse than being bogged down waiting for large files to open and close in a graphics editor. If you can afford it then implemnt a 2 drive RAID 0 array using either SATA-2 drives or Ultra SCSI 320 Drives for your working hard disk array. The choice of drives is also quite critical so do your homework in this area.
5) CPU Selection. Really now a simple question of AMD Vs Intel as Apple have finally realised the error of their ways and gone to Intel CPU's :devil: Dual processors (Intel D series, AMD X series) really kick butt when using PhotoShop.
The bottom line:
Building a quality graphics editing computer is a holistic process, there are a number of critical components and their impact on the whole performance is sufficiently critical that they cannot be singled out and treated as the single most important component. Items that are less important, motherboard, PSU, Case and so on.