There are three general classes of problems you may want to troubleshoot:
Troubles with the real drive
In my experience, this is the most common type or problem - something
seems wierd with the drive, and upon investigation, in turns out that the
pseudo drive is faithfully reproducing odd behavior occurring on the real
drive. So, if at all possible, try the same operations with the actual
tape drive, and see if you get the same results.
Trouble with tclient and tclientd
These can be diagnosed by putting tclient and/or tclientd in the debug
mode. This can be done by passing the 'debug' parameter when they are invoked,
or the debug status can be toggled on and off at any time by sending tclient
or tclientd a USR1 signal ('kill -USR1 .....'). The fields reported
in the debug output refer to fields in the devblok struct being passed
between tclient and tclientd.
Troubles in the driver
These are the hardest ones, of course, and the most important, since
any errors in the driver usually result in a system crash. There are
two main ways to try to see what is going on in the driver: the
special kernel printf, and the trace table.
You can use a special printf from the driver. The output from this
printf appears on the system console. For aix, you have to enable kernel
debugging (cf. 'bosboot -D ...') in order to get the output. Also, on
aix at least, use of this printf slows the system noticeably.
The device driver includes an event trace table. This is a wraparound
buffer that is always updated at signifigant places in the code. This is
especially useful in cases where the pseudodevice seems to be hung.
The trace table is read from the sdev device using a special ioctl call -
there is a program that will do that and format the results (syntax
'rtdiag /dev/sdev').