250 lines
14 KiB
Plaintext
250 lines
14 KiB
Plaintext
History - History of the DTraceToolkit
|
|
|
|
------------------------------------------------------------------------------
|
|
20-Apr-2005 Brendan Gregg Idea
|
|
For a while I had thought that a DTrace toolkit would be a nice
|
|
idea, but on this day it became clear. I was explaining DTrace to
|
|
an SSE from Sun (Canberra, Australia), who had a need for using
|
|
DTrace but didn't have the time to sit down and write all the
|
|
tools he was after. It simply made sense to have a DTrace toolkit
|
|
that people could download or carry around a copy to use. Some
|
|
people would write DTrace tools, others would use the toolkit.
|
|
------------------------------------------------------------------------------
|
|
15-May-2005 Brendan Gregg Version 0.30
|
|
I had discussed the idea of a DTrace toolkit with the Sun PAE guys in
|
|
Adelaide, Australia. It was making more sense now. It would be much
|
|
like the SE Toolkit, not just due to the large number of sample
|
|
scripts provided, but also due to the role it would play: few people
|
|
wrote SE Toolkit programs, more people used it as a toolkit. While
|
|
we would like a majority of Solaris users to write DTrace scripts,
|
|
the reality is that many would want to use a prewritten toolkit.
|
|
Today I created the toolkit as version 0.30, with 11 main directories,
|
|
a dozen scripts, man pages and a structure for documentation.
|
|
------------------------------------------------------------------------------
|
|
16-May-2005 Brendan Gregg OneLiners
|
|
I've been using the toolkit for a day now (wow!), and have noticed
|
|
a few problems I've been fixing. One of them was the dtrace oneliners.
|
|
I have them in two files, Docs/oneliners.txt and the examples in
|
|
Docs/Examples/oneliners_examples.txt. The problem is that when I'm
|
|
looking for a script, I'm looking in Docs/Commands - a list of the
|
|
seperate script files, or I'm doing an ls or find. Ok, so I've now
|
|
made each one liner a seperate script. This seems at first pretty
|
|
silly since they are oneliners and shouldn't deserve an entire script
|
|
each, but I've found having them as seperate scripts makes them far
|
|
easier to find and use. The scripts and man page for each script do
|
|
point out the fact that it's a one liner.
|
|
------------------------------------------------------------------------------
|
|
17-May-2005 Brendan Gregg Version 0.33
|
|
Version 0.33 with 33 scripts. Maybe I should make the version number
|
|
equal the script count. :) I just finished dtruss, dapptrace and
|
|
dappprof.
|
|
------------------------------------------------------------------------------
|
|
08-Jun-2005 Brendan Gregg Name changes.
|
|
I've renamed Docs/Commands to Docs/Contents. I found myself typing
|
|
"more Docs/Contents" by mistake a lot. ok, maybe it made more sense
|
|
to call it Contents after all. I've also made a symlink to it called
|
|
Index.
|
|
------------------------------------------------------------------------------
|
|
08-Jun-2005 Brendan Gregg Version 0.35
|
|
Version 0.35 with 35 scripts. Also touched up procsystime and some
|
|
man pages. Added the CDDL version 1.0.
|
|
------------------------------------------------------------------------------
|
|
09-Jun-2005 Brendan Gregg Version 0.42
|
|
Added 7 more scripts.
|
|
------------------------------------------------------------------------------
|
|
14-Jun-2005 Brendan Gregg Version 0.57
|
|
Added heaps of new scripts. Now at 57 scripts.
|
|
------------------------------------------------------------------------------
|
|
17-Jun-2005 Brendan Gregg Version 0.61
|
|
Restyled many commands.
|
|
------------------------------------------------------------------------------
|
|
28-Jun-2005 Brendan Gregg Version 0.70
|
|
Added several commands including dexplorer. Developed a few useful
|
|
variants of classic scripts while writing dexplorer, and have added
|
|
them to the toolkit (I kept wanting to run them individually but
|
|
not have to run an entire dexplorer).
|
|
------------------------------------------------------------------------------
|
|
25-Jul-2005 Brendan Gregg Version 0.77
|
|
Added tcpsnoop.d, tcpsnoop, tcptop. Because of their addition I have
|
|
dropped tcpwbytes.d and tcpwlist. These are complex scripts, but they
|
|
track TCP in an accurate manner. However! also because they are
|
|
complex scripts, I expect they will require maintainence for newer
|
|
versions of [Open]Solaris, as various probes may change. They will
|
|
become much more stable once a network provider has been added to
|
|
DTrace (which may be some time away).
|
|
Also added iotop, and updated a bunch of scripts. A lot of work went
|
|
into this version, although the version change doesn't reflect that
|
|
(I'm still keeping the version number == to number of scripts).
|
|
Also added rwsnoop, rwtop, and more.
|
|
------------------------------------------------------------------------------
|
|
26-Jul-2005 Brendan Gregg Version 0.82
|
|
Many new scripts added, many updates. This is a major release.
|
|
------------------------------------------------------------------------------
|
|
17-Sep-2005 Brendan Gregg Version 0.83
|
|
A few scripts have been updated so that they work better.
|
|
execsnoop, iosnoop, opensnoop and rwsnoop will be more responsive
|
|
(increased switchrate).
|
|
------------------------------------------------------------------------------
|
|
22-Sep-2005 Brendan Gregg Version 0.84
|
|
Some updates, fixed some bugs (cputimes, cpudists). Added cpuwalk.d.
|
|
------------------------------------------------------------------------------
|
|
15-Nov-2005 Brendan Gregg Sys Admin Magazine
|
|
Ryan Matteson wrote an article on the DTraceToolkit which has been
|
|
printed in Sys Admin Magazine, December 2005. It's quite good,
|
|
and made it as the feature article - which means it will be available
|
|
online for some time. Thanks Matty, and Sys Admin Magazine!
|
|
"Observing I/O Behavior with the DTraceToolkit"
|
|
http://www.samag.com/documents/sam0512a/
|
|
------------------------------------------------------------------------------
|
|
01-Dec-2005 Brendan Gregg Version 0.88
|
|
Many scripts were updated. Added the Apps category. I had planned
|
|
to add some key scripts, but they haven't made it out of testing yet.
|
|
------------------------------------------------------------------------------
|
|
03-Dec-2005 Brendan Gregg Version 0.89
|
|
Added nfswisard.d, fixed a minor bug with tcp* tools (see
|
|
dtrace-discuss mailing list).
|
|
------------------------------------------------------------------------------
|
|
12-Jan-2006 Brendan Gregg Version 0.92
|
|
Added a few scripts including rwbytype.d. Fixed several issues.
|
|
------------------------------------------------------------------------------
|
|
09-Apr-2006 Brendan Gregg Solaris Internals 2nd Edition
|
|
In the past few months I have been contributing to Solaris Internals
|
|
2nd Edition. This book (now two volumes) is really amazing. The 2nd
|
|
volume does use the DTraceToolkit where appropriate, and covers loads
|
|
of useful topics. While writing and reviewing material for Solaris
|
|
Internals, I've had numerous new ideas for DTrace scripts. Not only
|
|
that, but a few people have managed to send me well styled, carefully
|
|
tested, well considered DTrace scripts for inclusion in the toolkit.
|
|
------------------------------------------------------------------------------
|
|
20-Apr-2006 Brendan Gregg TCP bug fixed
|
|
Stefan Parvu sent me a bug for the tcp* scripts: on build 31+ they
|
|
error'd on the symbol SS_TCP_FAST_ACCEPT. This symbol was
|
|
renamed to SS_DIRECT (I checked the code, they are used in the
|
|
same way). Ironically, when I first wrote the scripts I had hardcoded
|
|
the value 0x00200000, then rewrote it "properly" by importing
|
|
the header files and using the symbol name. Had I been lazy and left
|
|
it hardcoded, the bug would never have eventuated. Not to worry,
|
|
it has returned to being hardcoded, so that it works on all builds
|
|
(until something else changes).
|
|
------------------------------------------------------------------------------
|
|
21-Apr-2006 Brendan Gregg Restyled - again!
|
|
I've been writing the "DTraceToolkit Style Guide", to document
|
|
the style that these scripts obey. It is quite strict, and sets
|
|
the bar fairly high. I've been warned that it may cause very few
|
|
people to ever contribute scripts, which is fine. At some point
|
|
I'll carefully explain the mentality behind this, but in a nutshell:
|
|
Users on critical production servers expect the tools to be
|
|
accurate, carefully tested, and cause no undocumented harm.
|
|
------------------------------------------------------------------------------
|
|
22-Apr-2006 Brendan Gregg Docs changes
|
|
The "Contrib" file was merged into the "Who" file. In hindsight
|
|
it is better to keep this data together than to split it up.
|
|
------------------------------------------------------------------------------
|
|
24-Apr-2006 Brendan Gregg Version 0.96
|
|
The toolkit now contains 104 scripts, however I'll keep the version
|
|
number < 1.00 until the dust has settled on these new scripts.
|
|
There is some special significance with version 1.00, it would
|
|
imply that every script had been tested for some time - not that
|
|
I've just added a few.
|
|
There is a new main directory, FS for file system related scripts.
|
|
There are some interesting scripts in there, from or based on
|
|
Solaris Internals 2nd ed, vol 2.
|
|
------------------------------------------------------------------------------
|
|
30-Sep-2007 Brendan Gregg Version 0.99
|
|
It's been a year and a half since the last release, and a lot has
|
|
happneed. Firstly, the DTraceToolkit has featured in the Prentice Hall
|
|
book,
|
|
Solaris Performance and Tools
|
|
DTrace and mdb techniques for Solaris 10 and OpenSolaris
|
|
|
|
written by Richard McDougall, Jim Mauro and myself. It is a companion
|
|
book to "Solaris Internals 2nd edition" by Richard McDougall and
|
|
Jim Mauro. If you are serious about becomming a DTrace guru,
|
|
especially on Solaris, then please study both books. (Yes, I realize
|
|
that many people are using the DTraceToolkit because they don't have
|
|
the time or don't want to become DTrace gurus; well, so long as
|
|
you are using DTrace anyway :). The performance book was a great relief
|
|
to write - since we were able to put to print much performance wisdom
|
|
and knowledge that was begging to be documented.
|
|
|
|
Then, in late 2006 I joined an advanced products engineering team
|
|
at Sun in San Francisco, a team which includes the three members of
|
|
team DTrace. It's been a great opportunity to learn from such
|
|
engineers, and to contribute more directly to DTrace. So far my work
|
|
has included writing a JavaScript provider, integrated inet_ntoa()
|
|
style functions into DTrace, and prototying DTrace IP, TCP and UDP
|
|
providers.
|
|
|
|
Working on the network providers is good news for the DTraceToolkit,
|
|
as it will indirectly help the tcp* scripts become more stable. Yes,
|
|
those scripts have broken a few more times during the last 18 months,
|
|
sorry about that, and it will keep happening until we have stable
|
|
network providers. This is why I only ever wrote three tcp* scripts,
|
|
and not at least a dozen, which I'd really like to do.
|
|
|
|
I did leave my pile of old SPARC and x86 development servers behind
|
|
in Australia, and brought over a couple of laptops. That has made me
|
|
more dependant on Stefan for testing the toolkit - especially on SPARC.
|
|
|
|
So, it's been about 18 months since the last release, which is
|
|
mostly due to having less spare time due to moving countries and
|
|
learning a new job.
|
|
|
|
Michelle from Sun docs has been asking for a newer version of the
|
|
DTraceToolkit for the OpenSolaris starter kit, which is why I'm
|
|
releasing this version now and not waiting a few more weeks as
|
|
I complete bug fixes.
|
|
|
|
So the good and the bad news for this release, starting with the bad,
|
|
|
|
Bad: tcpsnoop/tcptop still don't work on some Solaris 10 releases.
|
|
I've added versions that should work on Solaris Nevada and OpenSolaris
|
|
for releases from around late 2007. They are likely to break again.
|
|
The real answer, as always, is for stable nework providers to be
|
|
integrated into Solaris.
|
|
|
|
Many of the exciting new language provider scripts in this release
|
|
currently require downloading, patching and compling of the language
|
|
interpreter to get working. See the Readme file in each directory
|
|
for pointers.
|
|
|
|
Good: many more scripts to cover the new DTrace language providers
|
|
that are available (the DTraceToolkit is now 227 scripts). Many
|
|
updates to the Notes directory. Bug fixes. Some new categories
|
|
other than for scripts: Code - for simple programs to DTrace (and
|
|
for the example files), and Snippits - for useful lumps of DTrace
|
|
code to copy-n-paste from. The man pages are also making room
|
|
for documenting both stability and supported operating systems for
|
|
each script - now that DTrace exists for MacOS X Leopard, the
|
|
DTraceToolkit will begin supporting multiple operating systems.
|
|
|
|
This can be thought of as a developer's release of the DTraceToolkit -
|
|
to help people start using DTrace with Perl, Python, Ruby, Php, Java,
|
|
JavaScript, Shell and Tcl. I've written about 15 scripts for each
|
|
language, to cover the basics and to show the way for deeper analysis.
|
|
The scripts are also similar from one language to another, having
|
|
devoleped a tried-and-tested group of scripts for analyzing real world
|
|
issues - it made sense to repeat these scripts for every language
|
|
possible. To see what I mean, try reading,
|
|
|
|
Examples/j_cputime_example.txt Examples/py_cputime_example.txt
|
|
Examples/js_cputime_example.txt Examples/rb_cputime_example.txt
|
|
Examples/php_cputime_example.txt Examples/sh_cputime_example.txt
|
|
Examples/pl_cputime_example.txt Examples/tcl_cputime_example.txt
|
|
|
|
You might notice that the example files are more clearly and carefully
|
|
explained. Claire (my wife), wrote close to one hundred of them for
|
|
this release while I focused on writing and testing the scripts.
|
|
Claire has worked as a SysAdmin and as an IT instructor, and is well
|
|
skilled at explaining relavent technical details. And she can spell
|
|
much better than I can. :)
|
|
|
|
The future: I still have many new scripts and some bug fixes in mind,
|
|
as well as generally improving the Notes and Examples provided.
|
|
Hopefully it won't be too many months before you see another
|
|
release. Check here for the lastest installment,
|
|
|
|
http://www.opensolaris.org/os/community/dtrace/dtracetoolkit
|
|
------------------------------------------------------------------------------
|
|
|