Ruby is now ludicrously fast.
Ludicrious JIT is a Ruby 1.8 JIT which uses the libjit library that was written for dotgnu pnet's JIT core, very similar to what rubydium was trying to do. But libjit's come a long way in the last 3 (!) years, so if you're a Ruboi I suggest you try this out anyway.
It is a testament to the efforts of Rhys, Aleskey and Klaus to keep the library truly modular, reusable and independent of pnet's internals.
--The biggest difference between time and space is that you can't reuse time.
-- Merrick Furst
So it has finally happened - a stable release of dotgnu's Portable.NET and Libjit, after nearly a year and a bit more. The critical thing about this release is that Rhys doesn't yet know about this yet. This is the first release we've done without rhysw's involvement - he's been last sighted in Greephone land. So rather than the usual place, you can get the packages from the savannah download area.
This is by far the most revolutionary release dotgnu has had. The entire x86 engine is now fully native JITted. The windows.forms has taken quite a lot of work. My special congratulations to Aleksey Demakov and Klaus Triechel - who have worked especially hard of late to get libJIT into the shape it is today.
And in other news, we've put up a couple of Google SoC ideas at gnu.org. I'm very much interested in people picking up the AMD64/ARMv4 libjit core, either individually or as a team (does GOOG allow that ?). Also great strides being made in WinForms land, Radek has just managed to get the SharpDevelop editor to run.
The SD code uses a fair bit of PInvoke code bits to draw the caret and such, which is totally disabled in this rebuild. But it is quite an achievement at this point to be able to run a GUI component of this complexity.
Awesome work all around folks !
--Time is the most valuable thing a man can spend.
-- Theophrastus
*DRUMROLL* I'm proud to announce the first commercial product to use dotgnu Portable.NET in a big way. Trumpf.de have brought out a pulse laser cutting tool which uses dotgnu. The embedded control device runs linux and uses dotgnu to power its control UI, running a Windows.Forms application on the touch screen. Here's how the embedded appliance looks like - isn't it purty ?
It is one of those days that you live for - when you can stand back and marvel at what has become of bits you twiddled. And congratulations to klausT, avd and radekp for supporting Trumpf. Of course, much heartier thanks to Trumpf for believing in our code, being a good customer and investing a good ten thousand dollars or more into the project, even more when you consider the number of patches from them directly. That kind of customer focus has essentially revived pnet from the ashes of its past into a new good groove of embedded goodness.
If want, you can take a look at the official brochure, but which doesn't mention anything about linux or dotgnu, sadly. There's more related stuff upcoming - I'm trying to get ARM support resurrected with the help of Pengutronix folks.
A solid year's worth of work has gone into pnet thanks to this and the next stable release is only four days away. This is the first release to include libjit support - adding a full x86 JIT to the engine core. The twin team of Klaus and Alesky have reworked the engine so much that I've been having trouble jumping back into the project without asking too many stupid questions. So, look out world, here comes 0.8.0.
Been a good five years - looking back.
--You know, I have one simple request. And that is to have sharks with frickin' laser beams attached to their heads!
-- Dr Evil, Austin Powers
On a cold dark day in November 2005, Trumpf took a risk and bet 9000 USD on dotgnu. These were two grants of 4500 USD each, one for libjit (i686) and the other for the corresponding JIT coder in Portable.net, which I've jokingly referred to as the Southern "Summer of Code" (update: skwashd reminded me that he coined the phrase).
As of October 1st 2006, the work is considered to be complete. Congratulations to klausT, aleksey and kirill, who will all receive an equal cut of the pie. But it is not really about the money here, it is about the true spirit of FOSS - how people stand up to be counted, are put to the test and come out winners. I've been there, done that and still miss the rush.
Moving onto the bad news section of things, I'd like to take advantage of this occasion to formalize my sabbatical in DotGNU land. I've been promising to huff and puff for a while, but nothing's been blow away so far, other than the odd one-liner somewhere. So at least for a couple of months, I'll just stop promising stuff and go into "do & commit" mode of operations.
If that doesn't work out, I'll just have to be promoted to management or taken out back & shot.
--The reward of a thing well done is to have done it.
-- Emerson
Last october, Radek Polak suddenly announced that now dotgnu has a debugger. As much as that was good news, it still didn't fit in easily with the current design where the debugger works over the wire. On wire debug protocols are all the rage these days and for good reason, except that a few too many exist in some places. The new debugger backend is checked into Portable Studio SVN for those interested in the nitty gritty details. Here's a sample debug session.
The server is running at port 4571
Waiting for a connection...
Connection accepted
--------------------------------------
Shortcuts for recently used commands
0. watch_method PrintFn
1. unwatch_method PrintFn
2. list_threads
3. unwatch_all
4. stack_trace
5. print_locals
6. dasm
7. watch_location simple.cs 2
8. watch_all
9. watch_method Main
--------------------------------------
> l
<DebuggerResponse>
<Breakpoint Offset="-1">
<Location Linenum="0" Col="0"><SourceFile Filename="" /></Location>
<Member MemberName=".ctor" Owner="$Synthetic.$164">
<MemberSignature Language="ILASM"
Value="public hidebysig specialname rtspecialname
instance void .ctor(int32) runtime managed "/>
<MemberType>Constructor</MemberType>
</Member>
</Breakpoint>
</DebuggerResponse>
> print_locals
<DebuggerResponse>
<LocalVariables>
<LocalVariable Value="10" />
<LocalVariable Value="0" />
<LocalVariable Value="0" />
</LocalVariables>
</DebuggerResponse>
That is the raw debugger protocol dump, now I'll just wait for Radek (or someone else) to slap a pretty UI on top of that. In all probabilty, Portable Studio will let me debug dotgnu code without all the hassles we've been going through for the past few years.
Developer tools are the crack cocaine of the software world. One try and you're hooked.
--Debugging is over when people get tired of doing it.
Tom Tromey has put up information about a function GCJ backend interpreter using libjit. He's not put up the code as such, but here's the blog entry explaining what he's done. But a little digging found me this gcj-jit/jit.cc which could be the Real McCoy.
This is all really awesome for libjit because it adds another good use as well as a lot more test cases. More news later.
--Your motives for doing whatever good deed you may have in mind will be misinterpreted by somebody.
Finally, Rich333 appeared back on #dotgnu - of course, while I was away.
<scandium> and a few days ago I think t3rmin4t0r said that he's going to look at it to get
the interpreted mode of libjit right
<Rich333> so the chances of him helping with libCrayons instead of making smug blog comments
about it are zero... perfect
<Rich333> thank the gods
So let me make yet another smug comment about libCrayons while I'm at it. Heh, just kidding. I've been slacking for a while lately, I've been working on XPath which nobody around really needs that badly. Richard has been hacking away like mad on the crayons code. I just did a diff of the commit he made yesterday.
libCrayons/ 2006-03-06 +6774 -2673
I'm sure that he's been under-appreciated for all the effort he has put into pnet. Praise, where praise is due... [in other words - w00t, Rich333 r0cks]
Also if anyone's interested in helping (come on !), just give #dotgnu a raise and see if you're any good at low level graphics magic. You'd probably learn more about how UI systems work in two months than years in a classroom. And that's a promise.
--The trouble with opportunity is that it always comes disguised as hard work.
-- Herbert V. Prochnow
Even though libCrayons is taking its sweet time, the drawing library on top of it is actually moving along pretty nicely recently. Brubbel has been working quite hard on the path handling (as the changelog shows). He checked in an absolutely brilliant sample which should show off what pnet's graphics drawing is actually capable of. I've pulled out the code out of the FormsTest for the sake of those who just want to play around with it - TigerPath.cs.
The anti-aliasing and fill curve patches just rock, just take a look at the state the codebase was in before. Three patches in CVS and it is all smooth as silk - *sweet*.
It isn't often a huge patch hits pnetlib - but 600 odd lines in one week sort of makes me happy to just sit back and look at the pretty pictures :)
Now, I'm headed deep to investigate what XmlNodeReader really does...
--These are merely lines to me, but they paint a picture for you.
Actually, it happened before 2006 dawned. I was too busy to sit down and write about it. But it is official folks, the first peice of C# code has been run using the libjit x86 JIT. Here is the announcement.
If you would like to help out dotgnu - by talking about it, by testing it - any way you can, contact me (t3rmin4t0r on #dotgnu!irc.freenode.net). I really need someone who can fix up the XML parts of dotgnu. So if you want to help, be sure to talk me to me. Or talk to _Simon, he'll probably tell you what all bugs that I am slacking on.
--All one has to do is hit the right keys at the right time and the instrument plays itself.
-- J.S Bach
There was this thesis that SuperIlu was doing. The general aim was to study if we could freeze a virtual machine on one machine/cpu and migrate it to another - and now we understand a lot more about acorn nut burying squirrel migration patterns than we ever did before. Did you know that evil squirrels say we own your nuts just before they ship you off to the nut mines ? Amazing things they can do these days ...
The jokes aside, it is an excellent example of how Portable.net can be made to do things which it wasn't designed for - in a sort of Model-T driving a flour mill sense. The engine can be stopped ,moved to another machine and the whole thing will keep ticktocking as if nothing happened at all.
If you can read German, you can look at the real thesis. Or if you are like me you can read his implementation notes, which are thankfully in English. for the
--Hackers are just a migratory lifeform with a tropism for computers.
One of my slide ideas when explaining lightweight drawing toolkits for one the talks I submitted for lb/2004 had an architecture of System.Drawing and explained how a properly done up lightweight toolkit can render widgets on paper with a crayon. Nearly 16 months later, it is slowly happening - though not on paper.
Drawing logos, reminds me ... There was this one which I draw way back when I started on pnet. Nobody said it was bad, but everyone just ignored it.
If you are interested in working on libCrayons, wait till the christmas - new year is over. In all probability, the lead for that is waking up with a hangover at his apartment right now.
--If God has meant us to write, he'd have given us hands.
Oh, wait...
Highlights were discussions on libjit.
21:56] rhysw (n=rweather@CPE-149-167-179-155.qld.bigpond.net.au) joined #dotgnu.
[21:56] <t3rmin4t0r> meeting in 8 minutes
[21:57] <rhysw> hi everybody!
[21:57] <t3rmin4t0r> hi doctor ned
[21:57] <rhysw> :)
...
[22:07] <avdicius> so do we have only one jit_cache_method per function ?
[22:07] <rhysw> I believe so
[22:08] <avdicius> So we need to change the misleading comment in jit_cache.c
[22:10] <rhysw> as to the finally/catch question, have a look at
pnet/engine/verify_except.c - it generates the right code for
each block, with the right finally calls. When execution jumps
off the end without a match, it should call
jit_insn_rethrow_unhandled to throw the exception up to the next level.
...
[22:21] <t3rmin4t0r> newl: nope, the ILJITCoder stubs
[22:21] <KlausT> it's build with a configute switch :)
[22:21] <KlausT> configure
[22:22] <KlausT> --enable-jit
[22:22] <newl> neat
...
[22:31] <avdicius> rhysw: ok, thanks. Another question is about copying
1-byte data between memory and register.
[22:32] <avdicius> We could use in this situation only y ax, bx, cx,
and dx registers. The si and di do not work.
[22:34] <rhysw> Use x86_widen_membase to load the value and widen it
to a full word - then you can use any register.
[22:34] <t3rmin4t0r> avdicius: load at COP_BLOAD on unroll.c
Of course, Christmas is in the air ... but a lot different for people involved
[22:40] <rhysw> festivus baby - get out the pole!
[22:40] Action: KlausT wishes there was heat ... it just started to snow
[22:41] <t3rmin4t0r> KlausT: always greener on the other side
[22:41] <KlausT> lol ... yep :)
[22:41] <rhysw> Try sitting through an Aussie Christmas day, with 40C heat, and you'll just love snow!
[22:41] <t3rmin4t0r> and by that I mean, really the other side of the globe
[22:42] <rhysw> as my grandmother used to say: you can always put another
jacket on in winter, but there is only so much you can take off
in summer before you get arrested
[22:43] <scandium> lol
[22:43] <newl> rhysw: you forgot to mention that they're normally held at the beach. :)
[22:43] <avdicius> A few days ago I could not start my car because it was frozen
[22:44] <newl> That's always a good excuse to NOT go to work. :)
Had to quit early for the blug meet and a pretty boy photo (hair oiled up and clean shaven... heh) for my visa ;)
[22:54] <aathitude> i was looking at arrays in dpas
[22:54] <aathitude> the way i am doing it i am ending up using jit_insn_add and jit_insn_mul to compute the effective address
[22:55] <aathitude> the reason is this..
[22:55] <aathitude> jit_insn_load_elem_address uses the type paramter
passed to it
[22:56] <aathitude> for arrays with more than one dimension, sizeof(type)
will not amount to range_size*sizeof(elem_type)
[22:56] <aathitude> is it making sense?
[22:57] <rhysw> yes - multi-dimensional arrays will probably need manual
computation. The load_elem instructions are designed for the
common single-dimension case.
[22:57] <t3rmin4t0r> aathitude: like I said, you've got to calculate
offset = ((index1 * multiplier) + index * multiplier) + base_addr
[22:57] <aathitude> yes thats what i am doing now
[22:57] <t3rmin4t0r> finally, doing a load_elem with the final offset ?
[22:58] <rhysw> t3rmin4t0r: just what I was about to sugest :)
[22:58] <t3rmin4t0r> also there's the part about what to do when bounds
check fails :)
[22:58] <aathitude> that is what i am doing..
[22:58] <t3rmin4t0r> rhysw: when I meet you, I'm going to give you hell
for the Check2DArray in unroll_ptr.c
[22:58] <aathitude> yes bounds failure..
A productive one hour meeting... After a long time. Btw, got contacted by FSF for my christmas present. After all it is the season to give.
--'Tis the season to advertise
-- Calvin
The FPU unroller (+90, -85) is checked into CVS. Had some last minute issues with 64 bit memory address emitting while loading constants. That has been worked around by using the following.
#define md_load_const_float_32(inst,reg,mem) \
do { \
amd64_push_reg((inst), MD_REG_PC); \
amd64_mov_reg_imm_size((inst), MD_REG_PC, (mem), 8); \
amd64_fld_membase((inst), (MD_REG_PC), (0), 0); \
amd64_pop_reg((inst), MD_REG_PC);\
}while(0)
Technically speaking the following code should've just worked.
#define md_load_const_float_32(inst,reg,mem) \ amd64_fld((inst), (reg), (mem))
But the amd64-codegen.h does not actually accept 64 bit pointers in there. Which is really strange, because if it doesn't, I don't understand how Mono has a working AMD64 jit (same amd64-codegen.h in both places). The direct amd64_fld would be a lot faster than an indirect membase read. There are more small things left to do in there, like SSE/2/3 opcodes for speed. For example there is a convenient cvttsd2si to convert floats and doubles to integers without too much mucking around. Also I plan to implement trig functions directly using the FPU sometime in the future. Here's an old patch of how it could be done for x86/amd64.
The float benchmark has gone up 3x approximately. From a November snapshot of the engine giving 540 on floats, it has gone up into giving 1671. I think that a further 50-60 should come in as soon as I go SSE on this codebase. And for the first time ever, my entire pnetmark has gone beyond 10,000 - 10257.
And now I have no excuse for not hacking libjit ;)
--Eat flaming death, you minicomputer mongrels
There are a few things you can do when you're depressed. Some people take a walk, others drink, some look at old albums and a few jump out of buildings. Yesterday night (or like someone else pointed out: last night), I took a trip down memory lane. Trying to remember the happy times we had in the initial days of dotgnu. Looking back these were one of the best damned weeks of my FOSS programming, but while I was there - the suspense was killing. It is for weeks like these I keep hacking on dotgnu, one of these days we'll roll out libjit and it will be another Eureka! moment.
The public archives of these mails were missing from dotgnu, I'd got these backed up on a CD when I left home. ( update: I found them on gnu.org). There are quite a few more private mails with even more desperation in them, but I can't reveal them with a clear conscience.
To start off with on 27th Oct, Rhys decides to announce that he's going to go stop working on dotgnu & portable.net.
Date: Sun, 27 Oct 2002 07:15:04 +1000 From: Rhys Weatherley <rweather@....com.au> Subject: [DotGNU]Rhys's Role To: developers@....info As some of you are aware from the recent IRC meetings, I have been re-examining my role in Portable.NET and DotGNU. This has created a flurry of so-called "crisis discussions". The only "crisis" is that I am slowly burning out, and need to take some time off to think about what I want to do next. My doing so does not "kill DotGNU" as some think. There is a strong community here, but you all have to understand that no one person can carry this on their shoulders alone. I will be putting out a new pnet/pnetlib release in the next couple of days, to act as a check-point on progress so far. After that, I will be reducing my coding role until I figure out what direction I personally want to go in next. Others in the group should see this as an opportunity to take responsibility for a large piece of DotGNU and then commit to seeing it through. Cheers, Rhys.
It gets all kinds of responses. A guy named Jonathan Springer gets hacking. Then the weekend warriors happened. I sent in some ideas.
Date: Sat, 26 Oct 2002 19:48:44 +0530
From: Gopal V <gopalv82@....net>
Subject: [DotGNU]Forking pnetlib and moving on
To: DotGNU Mailing List <developers@....org>
Hi Everyone,
I've been thinking about our dependence on Rhys to hack the
compiler. DotGNU Pnet should be a community and the current state
of pnetlib has been unsatisfactory for me .. The coding standards
of pnetlib has made us straggle in comparison to mono or ocl....
As the lead developer, I find it very disappointing to reject
patches from well meaning developers for reasons of stylistic
content or code quality ..
Get Pnetlib building with CSCC
------------------------------
Let me get this straight, this is my ultimate and total
aim right now . I am going to solve the quibbles about pnetlib
not compiling with CSCC for once and for all times. How many
developers have we lost , who would have taken our side if
we had been building pnetlib with pnet.
This mail is almost `too little,too late` in content, but
hear it out , as it is the only chance we have of surviving the
impending crush ... We're almost to the top of the hill and the
climb's getting steeper.
Compiling pnetlib as an aim might seem to be too obvious ,
but it is not ... It is the only way I can get what I want . The
good or the obvious solution to the problem is fixing cscc ..
But unfortunately we lack the skill to proceed on it in a short
period. The bad solution is to modify pnetlib to make it work
with cscc . This might be bad , as we might decide to remove
to changes in the code at a later date.
....
I would like to call a meeting of all the good people of DotGNU
Portable.Net at 1900 hours UTC to discuss this idea...
I had been to the point of giving up on this , what with all my personal
problems. But I can't desert with a clean conscience ... and I NEVER
QUIT !!.
Forking might be an extreme measure, But It's time to get down and dirty..
Gopal
--
The difference between insanity and genius is measured by success
IRC flys with discussions on how to fix bugs and three or four people talking at the time. We aren't that hopeful on fixing the bugs. But I'd put in a year of solid work into that project, I was going to go down fighting. And suddenly it starts working, there are no errors during compilation, but the binary is invalid. But who cares - it WORKS !!!
Date: Mon, 28 Oct 2002 20:05:28 +0530
From: Gopal V <gopalv82@....net>
Subject: [DotGNU]Pnetlib compiles and RUNS pnet/samples (was: Forking pnetlib
and moving on)
To: DotGNU Mailing List <developers@....org>
(- I'm writing this is in a half dream delirium , please
ignore the nonsense as needed -)
If memory serves me right, Gopal V wrote:
> Get Pnetlib building with CSCC
> ------------------------------
>
> Let me get this straight, this is my ultimate and total
> aim right now .
> ... We're almost to the top of the hill and the climb's getting steeper.
Since sending this mail I clocked 18 hours before my little box here ...
and I'm sure Jonathan Springer has done none less...
But this is official, folks
*drumroll*
"PNETLIB COMPILES WITH CSCC" !!....
And I'm not talking about "Just compiling it", I even managed to get
all the pnet/samples running just as it should !!!.
Sure there are errors all over the plce , but does that matter ?...
Not a bit .. pnetlib compiles !!! WOOOHOOOO ....
Wait a minute, I didn't fix cscc all the way ... But changing 5 lines
to compile 27,572 is a good bargain ... A lot of codegen bugs remain
still and we'll tackle them together ! ... So Who's moving in ?.
After that, me and springerjp we've fixed all the problems in the codegen and compiler to build pnetlib without any verification errors. With an appropriate anti-climax :)
Date: 01 Nov 2002 11:26:26 -0500 From: Nick Zigarovich <nick@....org> Subject: [DotGNU]pnetlib verification To: DotGNU Developers <developers@.....org> <t3rmin4t0r> hmm... my email provider seems to have DNS problems <t3rmin4t0r> "CSCC has compiled pnetlib without any verification errors" ... Hooray
If you didn't know, rhysw was here for two years after that. All goes on to prove that it is Darkest before dawn. That was one of the good days, when we worked hard and we won. It was worth a good fight and we gave it one - three years later, I'm still proud of that one day. I probably always will be.
This is why I hack. For the rush to know, feel and enjoy the fact that you made a difference. One of those beaches of code where you left your footprints till the next tide of rewrites wipes them away.
--The difference between insanity and genius is measured by success
Need I say more ? That 1700 USD pays for my air tickets with enough buffer for exchange fluctuations. Now I have to clean up my act and get my visa papers finalized. Monday is going to be a very busy day indeed.
--if tcp was less formal, syn/ack probably would be WAZUP/Y0
Dotgnu has a new task at hand and it is worth 4 lakhs to whoever would care to do it. The task is simple and straight forward, you need to write a fast i686 JIT for dotgnu. Half the task is already done in libjit - which has around 65 KLOC of code. All you need to do is finish the x86 engine in there, with the help me and many other dotgnu folks around the place.
