[These are notes from a session 4D Summit 2008.]

Presenters: Laurent Ribardiere, Laurent Esnault, Olivier Deschanels, 4D SAS

Laurent Ribardiere

What's changed since last year's Summit...

LR wanted to start by clarifying something about blob and text storage outside the record. There are times when it's a negative since when you have to search that field it's something like twice as slow. But then again if you don't need to do that the record is much smaller and more efficient. You can set the threshold as to when it's stored in the field and when it's stored outside.

Auto increment is much better than using a trigger for set key values.

TRUNCATE TABLE is extremely efficient way to dump data - something like 1000 times faster than DELETE SELECTION. They've optimized DELETE SELECTION so it uses TRUNCATE TABLE in certain cases. But do watch to see if it's failed since if it has problems with one record it does nothing. Also, it won't work in transactions.

The joins and how they work in QUERY BY FORMULA were improved in v11.2. You can now do things like:

QUERY BY FORMULA ([Employee];[Employee]CompanyID=[Company]ID & [Company]Name="4D")

A join is performed, there is no need for a structural link.  If you leave out the [Employee]CompanyID=[Company]ID a structural link is required. This may create problems because you may have intended to use the value of [Company]ID, not specify a join relationship.

The cache manager in v11 is much more efficient. No compaction is needed so it's far more scalable and can use non-contiguous memory. 4D v11 is a 32-bit application, so the theoretical limit is 4GB. The practical limits (from testing) are 1.5 GB on 32-bit Windows, 3.7 GB on 64-bit Windows (though not recommended to go that high), and 2.7 GB on OS X 10.5 (Leopard).

Even though 4D is a 32-bit app, there are benefits to running under a 64-bit system - namely the cache size can be larger. Basically under a 32-bit system you get a total 4 GB that can be allocated to 4D, whereas under 64-bit you get up to 16 GB with some things being stored outside the 4 GB that's used directly by 4D. A 64-bit version of 4D is coming... Then you'll get a total of 32 GB of RAM that can be used for 4D.

WAN performance for client/server is dramatically increased. Fewer requests and larger block sizes are the way it was achieved. QUERY went from 4 commands to 1, READ ONLY went from 1 command to 0. He did a demo on v2004 that took 34 seconds, on v11 it took 6 seconds.

Displaying a selection of records has also been improved over a WAN. v2004 had improvements over v2003, but v11 takes them further. The entire page is returned in one request and only the data displayed are returned. Listboxes are even more efficient than output forms because there are fewer tricks to get them to work.

Sets and named selections stick to where they are used (client or server). $ or <> are now used for namespace only. You could have a $ set on the server and a <> on the client (only). 4D knows when it needs to transmit back and forth and only does it when necessary. It's all transparent to the user and the developer.

Laurent Esnault

Client/server is now mostly based on polling, but on push. This is true in the development environment as well. Both the client and the server push as needed. For example the server could push an EXECUTE ON CLIENT out to the client.

You can log all the requests received by the server. This includes the Application server, DB4D Server and SQL Server requests. Activate it by doing SET DATABASE PARAMETER (4D Server Log Recording (28)) or "Start Request Log" in the 4D Server console. Olivier showed the logging action with a tail -f command in OS X...

QUERY BY FORMULA and ORDER BY FORMULA were previously discouraged in v2004. In v11 they're fine to use. It's now just one request to the server - it's no longer sequential over the network. Variables used directly in the formula are evaluated on the client. Variables used inside a method are evaluated on the server. Be careful that this may break compatibility - but there's a checkbox to make it act like v2004. If you need to turn this off and on dynamically use SET DATABASE PARAMETER. Compatibility mode is still significantly slower, but it's much faster than before because they page 100 records at a time instead of sending one by one.

Triggers are executed on the server in the twin process. Twin processes share transaction state and record locks, however there is no more client/server synchronization for current selection and other table records. The twin processes now have their own set of process variables in compiled mode. The execution of triggers no longer blocks other processes - many triggers can execute at the same time. You can now do pretty much everything in a trigger that you can do in a regular process except change the current record.

You can use Execute on client to push information to a client. The function is more efficient than it was previously.

You now need to think about when you want to use Execute on server (stored procedure) and when you just want to use the checkbox Execute on Server that's now a property of all methods. Execute on server (stored procedure) starts an independent process on server. You may find the Execute on Server (property) to be more useful and just as fast because it shares your process variables as well as $0.

Think about using the Execute on Server property to avoid triggers.

But make sure you don't just always execute on server since it may overload the server and not make use of the processing power of the clients.

Laurent Ribardiere

In terms of the architecture. The application server (4D language code on server, port 19813) and the HTTP/SOAP/Web server are still cooperative, single threaded. However, the SQL Server (port 19812) and the DB4D Server (port 19814) are both using preemptive multi-tasking.

The point is if you need performance stick with preemptive threads, not cooperative threads. That means you should keep the amount of 4D code running on server to a minimum (including triggers). If you call SQL from a 4D method it's cooperative. If you call it as an external data source it's preemptive.

They're working on making everything preemptive.

Do realize that you only have one data file and possibly many preemptive threads. 4D's cache is important, even better is the processor cache, but when those aren't enough contention can be an issue. So the more RAM you have, the bigger 4D's cache, and the faster your drives, the better...

DB4D data access is cooperatively threaded in 4D mono but preemptive with 4D Server. This can make somethings much faster. He did a demo showing how big a difference this can make. He also showed 4D using multiple cores on an 8 core machine.

Tags: 4D Summit 2008, 4D-v11
Categories: 4D