My Personal Thoughts on 4D Summit
Last year's Summit saw 4D taking a giant leap forward with v11, but it wasn't all there (no 4D Server v11). This year's Summit was more about delivering for the real world and refinement.
4D Server v11 SQL really is pretty cool, and quite powerful. A lot of what it was about was discussed last year, but it's interesting to see cases where client/server is actually faster than a standalone/mono application. And most importantly, it's here now - not a demo of an unreleased product.
v11.3
Then there's v11.3 - If we were back in the v6.0, 6.5, 6.7, 6.8 days v11.3 would be v12 - it really does have some wonderful enhancements in terms of speed, SQL (much improved user management just to name one thing) and SVG (the new component). But the issue, IMHO, is that they didn't completely stabilize v11.2 before doing the enhancements so even 11.3 will probably be a bit of a gamble for a demanding, mission-critical app. At the same time, it's such a big upgrade it is going through a lot of QA, which is always good. It's also good that they said that there will be no new features for v11 after 11.3 - from here on it's all about bug fixes and making the product stable, though I can see them releasing new features as a component like they're doing with SVG.
4D Web 2.0 Pack v11 (the AJAX part of it)
Even though I'm all about using 4D on the web, and I know some of the guys on the 4D Web 2.0 Pack team personally, I haven't ever felt like I wanted to use 4D Web 2.0 Pack. That opinion changed at Summit this year for two reasons. 1) Brendan's mention of Javascript maturing to the point that incompatibilities aren't as huge as they used to be, and 2) Julien's presentation showed that 4D Web 2.0 Pack v11 really has matured considerably. They've had tools to help you build apps before, but their Green Tea IDE really was a big step in the right direction, though they did stress that it's just how you start your app - it doesn't handle everything you're going to want it to do.
Performance
That sorta brings up what was the main interest for me this year - performance. I have a 4D-based web site that gets over 1 million page views a month from real users, plus all the bot traffic (which is considerable - Googlebot alone is over 350,000 pages/month). The client wants to launch another site, integrated with the first one, that will be FAR larger. My rough estimates are that I'll need to support 50 million page views a month within 5 years (that's an average of 20/second). 4D's internal web server can't handle the load I have now, but I need 4D v11 to handle 50 times the current load.
After quizzing Julien Feason and Charlie Vass at a 4D Kitchen session, for high volume uses I need to avoid everything that hits the cooperative thread and hit the database engine as directly as possible - avoiding 4D code whenever possible. That means I have to avoid triggers and the 4D internal web server. Solutions like the AJAX part of 4D Web 2.0 Pack which are based off 4D's internal web sever are a problem - at least for high volume uses. That, and the fact that my client still has IE6 as their corporate standard and 4D Web 2.0 Pack doesn't fully support IE6 means that even though I now want to use AJAX part of 4D Web 2.0 Pack, my use of it will be pretty limited.
However, since you now can hit 4D's database engine directly there are plenty of other options and I'll be rewriting a fair amount of my web app in PHP which can use ODBC to hit 4D's SQL engine without going through 4D code.
4D for Flex (part of 4D Web 2.0 Pack v11)
While the AJAX part of 4D Web 2.0 Pack is a problem for me, 4D for Flex is a tool I can use, since it hits the preemptive part of 4D directly and it works just fine in IE6 (provided you have Flash 9, which as fate would have Flash 9 isn't part of my client's standard corporate build either). The session on 4D for Flex made it seem really complicated, but what I found most interesting about the session were the comments in the last 5 minutes during Q&A... First Tim said that he much preferred working with 4D for Flex than the 4D AJAX framework. Then one of the attendees chimed in saying once you get over the initial learning curve with 4D for Flex you'll love it and think of it as invaluable. He went further saying that the thinking when you work with Flex is more compatible with how you think in 4D than the thinking when working with AJAX.
The Language
One thing Laurent made very clear (without actually saying it) was that the current 4D language is dead - he literally called it "the old language" (even though there's no "new language" yet). If I understood things correctly, all those lines of code you've written in 4D will never execute in a preemptive multitasking manner. There will be a new language, it will be ECMAScript based (like Javascript and ActionScript), and that will be the language that will work in 4D's preemptive multitasking environment. What this means is that current 4D code, while it will be supported for a long time, is a bit of a liability. If you really want to get the most of 4D in the future you won't be running the code you have now.
From my perspective that means the more I do in PHP and Flex, the better. Remember, with PHP and Flex I can hit the SQL engine directly and use preemptive multitasking now, and the next version of 4D will have a code editor that supports editing PHP files, and a web server (based on Apache) that has a PHP processor built-in. So Laurent's comment last year about triggers being bad was probably more about the language used by triggers being bad. Still, getting away from the 4D language is a huge change in thinking for most of us since the current 4D language is like an old friend.
SVG
SVG was the pleasant surprise for me at Summit. I've been wrestling with EPS files that have emedded TIFFs. What I realized is that if I get SVG versions of those images I can do all sorts of things like swapping out the TIFF with a low-res JPG and even watermarking the embedded image and translating the text in the image from data.
Wrap-Up
So, from my perspective it was a useful Summit. 1) It helped clarify strategies to deal with the problems I have now. 2) It let me see that the primary remaining weaknesses in 4D (IMO) are being resolved in the next version.
Tags: 4D Summit 2008
Categories: 4D
October 13th, 2008 at 3:50 pm
Jay,
Great coverage again. Remember that all DB Queries irrespective of whether they are 4D based or SQL based do run as pre-emptive threads and can run across multiple CPUs/Cores. So I don’t think there will be much performance improvements by moving to a PHP solution as you think especially when you add ODBC driver overhead.
The only issue is when running native 4D Code on the server. For scalability reasons anyway I always think you should run with the web server running on 4D Client anyway (even on the same machine) and this would allow you much better scalability than going through an ODBC solution. You can always scale further by running multiple clients as web servers.
This would be an interesting benchmark scenario that we will do some tests on and report back the results.
Thanks
Brendan
October 13th, 2008 at 4:46 pm
Brendan - Thanks for your comment.
My experience has been that the internal 4D web server (v2004) can’t take that much load. It’s fine for most sites, but I started seeing pretty regular problems when I got above 1 million page hits a month, and by the time I got to 2 million page hits a month I was having serious problems until I implemented page caching that takes most of the load off 4D.
v11 will make that better because the underlying database engine is so much faster, but since the internal web server is essentially the same as v2004 the web server is still the bottleneck. I expect v11 could handle my current load, but not 25 times the load that I expect in 5 years.
Personally, I’ve never liked the idea of 4D Client for the web server. It feels like a hack, and there’s a real cost in hardware and hosting since the server is at a big colo facility. Plus, with “the writing on the wall” for current 4D code, a PHP approach seems more future proof.
The (theoretical) advantage of PHP/ODBC is that it’s a completely multi-threaded solution for read-only uses since no 4D code will be executed in read-only situations. There may be overhead with ODBC, but there are no bottlenecks, so it’s scalable.
That raises the question - Laurent mentioned that a JDBC driver could be written in about a month based on how 4D for Flex works. Can something be written for PHP that avoids the ODBC layer?
If you guys do benchmarking I’d love to work with you on it. It’s important to not just have linear loading models. You need to simulate hackers and rogue bots. I’m constantly hit with them and when the same machine requests a page 20-50 times in a second it can bring 4D 2004 to it’s knees (though it does recover, eventually). With the site being public, I’m dealing with sort of a wild west sort of scenario, not the polite intranet/extranet traffic most 4D web solutions support - hence my somewhat paranoid approach…
Thanks again for your input and help!
October 13th, 2008 at 9:42 pm
Jay,
Love to work with you on the benchmarking. I am sure you did note that we do have plans for a new web server architecture to address some of the issues you raised. So I think that this may take away some of the issues you are seeing.
Anyway I’ll ping you once we get to the benchmarking stage.
Take Care
Brendan
October 15th, 2008 at 6:49 pm
What’s the deal with multi-core pricing.. is this going to be configurable on the new 4D Store?
November 14th, 2008 at 9:01 am
Hi,
Since you are talking about PHP and 4D, I just want to ask why I am having a problem adding my DSN in Mac OSX. I want to use our existing 4D database with PHP that is why.
I hope I am not a bother
Best regards,
James
November 16th, 2008 at 4:17 pm
@James - I don’t use PHP to connect to 4D yet. I will next year, so don’t have an answer for you yet.
November 20th, 2008 at 6:30 am
If ever I will figure this out, I will give you a shout. Your blog is already in my bookmark…