I am worried!

by Kim 29. October 2008

I am seriously worried! Wake up call!

I perhaps should be worried about peace on earth, financial crisis, cost of energy, climate changes... and indeed I am... but thats not whats on my mind right now.
Whats on my mind is that Im seriously worried about the future of my beloved programming language Delphi. Since 1995, I have used Delphi extensively. Since 1998 I have contributed to the Delphi community via various projects. Since many years I have been a Borland/CodeGear Technology Partner. In other words, I have been living Delphi, breathing Delphi and dreaming Delphi... (sort of...Wink).

I have been following Borlands rollercoaster ride with Delphi being the rising language, setting the standards for how to program, thru the very, for Delphi and the community, unfortunate (lets call a spade a spade... stupid) decisions that Borland have made over the years. The constant uncertainty about the direction of Delphi. The rise of cool technology which gets to some level of adoption, after which its being dismissed from one day to the next etc. It certainly have been a ride where one have had to have a certain stamina and to some extent, naivity and optimism to stay on board.

After these initial mind travels, I think its in place to wish RemObjects congratulations with their agreement with CodeGear about the new CodeGear/RemObjects joint project with codename Prism. What is Prism? Its also called Chrome and later renamed to Oxygene. Its an Object Pascal variation designed from ground up for the .Net platform. It contains alot of cool features, not found in Delphi today. Because its close to being a 1st citizen development language on the .Net platform (programming language wise I would consider C# the only true 1st citizen due to the direct ownership and management by Microsoft that 'owns' the .Net platform), its also able to integrate with all the new cool features of the .Net platform... nearly anyway. Its using Visual Studio 2008 as the IDE. Actually I think its really something of an impressive feat by the RemObjects guys to lift this project to the level it has reached today.

Hence, despite RemObjects and C4D's rivalery on other areas, I clearly recognize the technical merits of their product Oxygene.

So all that aside, get to the point old man!

What have historically made Delphi a development tool of choise (the list may not be complete, add your own reasons)?

  • Fast one pass native code compiler
  • Good debugger
  • Integrated debugger, editor, compiler and designer (RAD)
  • Good structured language
  • Good backwards compatibility between releases
  • Generally well designed control and component framework (Visual Control Library - VCL and RTL).
  • Good price point
  • Very good community support
  • Not really any other development environments out there with the same featureset and usability.

In addition, slightly later in history, further reasons made it compelling to use Delphi:

  • One source Linux support via the product called Kylix.
  • One source .Net support via the product called Delphi/.Net.
  • One source Win32 support via the product called Delphi/Win32.

Hence people started out with Delphi on the Win32 platform and some eyed an oppertunity to move their applications or components to other platforms as they were made available. Even today some people use Kylix as the development tool to delivery their applications to the Linux platform, many years after Borland made yet another stupid (spade=spade) move.... to discontinue the Kylix project.

Later Java and specially Visual Studio.Net have made serious ripples in the RAD development waters, blurring the originally apparent advantages and image of Delphi. Both have gained alot bigger market shares in the corporate environments than Delphi ever had. Specially Java has been able to (to some extent) replace existing mainframe oriented programming paradigms.

On the desktop, new requirements related to graphics, 3D, animations and the user experience are constantly rising. These are areas where specially Adobe's Flex and Flash RIA solutions have gained a very strong foothold. A foothold I doubt even Microsoft will be able to seriously dent. JavaFX, Silverlight and WPF are technologies also competing in that area. And our beloved Delphi have very little ammunition to provide there, except for relatively disconnected 3rdparty component providers trying to provide some of that functionality in the Delphi environment.

So to sum up the situation.. Delphi has lost foothold in the Win32 desktop space. It has no foothold in the Linux desktop space. It has no foothold in the Apple desktop space. It also have no foothold in any mobile spaces. Further it has lost whatever foothold in the server arena, where it has never been really really strong, partly due to conservatism and later 'Noone is shot if a project fails when one have chosen IBM, SUN or MS for the platform/development tools'.

So whats left? Mostly small developers who still have the freedom to choose according to their hearts rather than their wallets or corporate policies.

If corporates want to develop new projects for the MS platform, what will they choose? C# is most likely the answer.

Will corporates choose to develop native Win32/Win64 applications for new projects if they know those are going to reside on the MS platform only? Most will probably not. They will choose to use .Net due to the hype and the maturity of the platform.

Hence Delphi's iceflake is really melting rapidly from all sides. And that sentence is the trigger for my now almost complete loss of faith in the Delphi environment as a future proof platform.
CodeGear may have chosen a superiour compiler for .Net by hooking up with RemObjects, but they have also ensured that all existing projects and components that have already been made available for Delphi.Net on the .Net platform now see no future with Delphi. If they want to exist on the .Net platform, they only have the choise to use Prism/Visual Studio.Net ... or C#/Visual Studio.Net. The safe bet is to use C# as it, without doubt, will be the most up to date development language for the .Net platform for as long as MS decides that the .Net platform have a life.

Net result is thus that existing Delphi developers, who have played on the horse of cross platform support using single source, now have to choose to create multiple source trees, or simply to go where the cake is bigger and the current new technology emerges from. And thats not in the native Delphi world.

I have said it for years... Delphi's absolutely biggest advantage was the feasibility of RAD cross platform deployment based on single source, and the originally very vast developer and 3rdparty vendor community supporting the product.

CodeGear have essentially chosen to cut the legs off Delphi or put differently, to put a flame torch to the ice flake on which Delphi stands.

Not providing single source, multi platform support, equals dropping the one single feature and unique sales point, that made Delphi stand out these days, and that could make small/medium sized corporations think twice before jumping in bed with C#/VS.Net or Java.

Its unfortunately a very depressing outcome, and not so full of optimism and joy, as many previous blogs about CodeGear, that I have made.

Cross platform, single source .... cross platform, single source .... cross platform, single source .... cross platform, single source .... cross platform, single source!
THAT WAS YOUR UNIQUE SELLING POINT - EMBARCADERO/CODEGEAR!

/Kim Madsen

 

 

 

Solong Borland.. it has been a true rollercoaster ride with lots of fun on the way!

by Kim 8. May 2008

Yesterday my long time good friend and colleague Richard pinged me with a weird looking web address: www.embarcadero.com

When receiving such an address, there must be a reason... could it have some funny jokes or ahrm... pictures... in it... Richard usually dont send such links to me.. (hmm... I wonder if he keeps those to himself Wink .....Just kidding Laughing). So it had to have something to do with what we do.

The name itself didnt ring many bells to me... but that can be because I have been living under a stone for a couple of years coding. Anyway.. I clicked it.

While the web page builds up I notice.. hmm... there is a cool tool... and there is another... but what have that to do with me or us?

Then suddenly a text written in relatively small font showed up... Embarcadero have bought CodeGear...  a few secs... WHAATT!!!! Is this April fools day?.. checking the calender... and year... nah... its indeed just any other normal day.

First impression were... a tools company not much different (although slightly bigger I admit) than us, bought CodeGear! How on earth can that happen. The mouse ate the elefant. We are DOOMED!

Well... then it was time to a second go on what exactly was Embarcadero (couldnt they have named it something pronounsable?.. and writable...my fingers are making knots trying to type it Smile).

Reading over what it was all about, then it shows that Embarcadero is a relatively big company, actually people wise bigger than CodeGear (but they have certainly not being making any fuzz about that in the media Im reading), and whats more important is that Embarcadero is just one of several private companies owned by a huge private investment company. Hence we are suddenly now talking about money backing of proportions that would make Borland have wet dreams in night.

Ok.. money and size isnt an issue anylonger... but what about the longevity of the CG tools? Could it be a shark attack that results in some technologies being digested and put into other tools, while the rest is discarded?

Only the future knows how it fares, but it certainly do look like Embarcaderos and CodeGears toolsets are extremely complimentary with no direct overlap at all, which indicates that there is a true potential for huge synergies between the companies. They have alot of dedicated and knowledgable developers, and we all knows that CodeGear have the same but on other type developer oriented projects.

Embarcaderos have an organization that contains all the relevant functions for running a developer oriented company, while CodeGear probably have been somewhat amputated in the seperation from Borland, and I could imagine that some administrative and marketing functions within CodeGear either were missing or at least severely limited in manpower. In that scenario, those areas would be naturally strengthened by this acquisition.

As Embarcadero have been targeting a very different vertical market than CodeGear, they also have their feets inside some places where CodeGear would never before have had access, specially related to native development (example Java houses and other non typical areas for CG native tools. Visa versa, Embarcadero now ofcourse have easier access to the many dedicated supporters of CG products, and this way will expect increasing sales of their own products. The total market for most people like us.. 3rdparty tools developers, does seem to open up as the market canvas have now been broadened and the exposure of our products potentially made alot more visible in usually non accessible places.

That should bode well for CodeGear customers too. If the market canvas grows, the potential to earn money for vendors targeting that canvas grows and that ensures stability and progress towards CodeGear customers.

My current take on the situation is that, despite the initially surprising selection of buyer, this is probably the best possible outcome one could dream of.

So ... Very welcome Embarcadero!

I think we all will have a great future together. Delphi, JBuilder, C++Builder now have both the code quality and the market canvas and exposure to really make things happen!

(I will try to get used to writing  "Embarcadero".... but boy... CodeGear is easier... will probably have to use the shorthand writing of it... CG vs ET... battle of the galaxians)

I have seen the light.. I have faith!

Kim Madsen
TeamC4D

Definition of outdated...

by Kim 15. March 2008

Warning... the following may contain provocative statements and irony.

There are often a reference to a programming language to be 'outdated', and often that happens in comparison to another language that someone wants to
promote.

Say.. Delphi... many places management and project managers refer to it as outdated.

But what exactly means 'outdated'? Why is something 'outdated'?

Looking strictly at the wording, then Java is more outdated than Delphi, as Java was invented in 1991 based on a C syntax, and Delphi in 1995 based on a Pascal syntax.

And buzzwords like Garbage Collection was invented all the way back in 1959 when Lisp was invented. Heck even Basic had GC built in.. so that surely _must_ be an outdated feature! ;)

Then there is C, which C# and Java and C++ et all builds parts of their syntax on... that was invented back in 1972... must seriously be outdated... why on earth are anyone still using C/C++ to make operating systems, real time systems and other applications that are critical and where leaks and severe bugs are no no's as they have a huge impact on the use of the product?.. yes.. why on earth is that.... reflection... ah.. reflection..

What about the term 'reflection'....

Reflection is in its core the same as RTTI in Delphi (RunTime Type Information), but given a new shiny brand.
It was already available in Smalltalk invented back in 1970/71.. so surely that feature must be perfect examplified object of being totally outdated.

Hmm.. object... object...     object oriented... that word combination rings a bell somewhere... now... we all know that C# and Java are masters of OO... anyway so are the claims... but did they invent OO?... nah... we all know that Bjarne Stoustrup invented C++ and the object oriented scheme dont we?..

Nope.. not even close. It goes all the way back to 1960s, and the language Simula that was invented by a couple of Norweigians.

So OO is really pretty old stuff.. it cant be useful today that old stuff can it? It must virtually be outdated!

That rings another bell... virtual... virtual.. something that isnt there for real!... weird word... ... virtual machine.. thats really a buzzword in this decade... What is actually referred to are process virtual machines. Things that enables on the fly interpretation of semi compiled source code (also called Pseudo code or P-code). Interpreter... that goes all the way back to 1930 (and perhaps even longer) by a numerical interpreter by IBM... THATS old.. so we wont even spend a startrek second more on that.. that is definitely outdated.

However there is another variant of process virtual machines, that use a technique called Just In Time compiling (JIT).. ie. the source or semicompiled source (P-code) is compiled to machine instructions shortly before the time the instructions are needed while executing the application.

THAT sounds cool... it MUST be a novel and very modern thing since it even got a modern name for it... Managed Code, right? Nah.. VisualWorx (which is a SmallTalk implementation) did that almost 30 years ago... in the 1980s. Thats old! Thats outdated!

Well.. you get the picture I hope...

What imo should be outdated is how money controls hype that controls management that controls pure logic and common sense.

Lets take a step back and discuss what exactly what is the reason for new commercially popular languages today?

My take is that its not technical, practical or even (for the single user) economical reasons that is the reason for them.
Its a global battle about marketshares and control. In other words... something happening in the very high altitudes where the air is so thin that people living in such places will get problems thinking rationally.

Kim Madsen
TeamC4D

The kbmMW transport(s) part 1

by Richard 13. January 2008

I’d planned this blog entry to be about the changes I’m making to the myc4d customer portal but over the past few days it has become clear that there is quite a lot of interest in our new kbmMW transports that are in development.  To save making the same comments over and over on the newsgroups I’ve decided to blog about what we’re up to and take the wraps off them instead.

 

There will be a lot of detail covered but some facts will remain un-answered for the simple reason that some things I do not know the answer to yet J

 

I’m going to do this in a semi-Q&A style to get my headings out.  Have fun! Tongue out

What is the new kbmMW transport?

 

Let’s kill this one off straight away.  The new transport is not a single new transport.  It is a set of transports built on top of our own socket layer removing the need to use third party communication stacks such as Indy or Synapse etc.

 

What is the motivation to build our own communications stack?

The motivation behind the project was driven by a variety of sources. 

 
  • There is growing discontent in our newsgroups about using the Indy components as the underlying communications mechanism.  Reasons given vary but within C4D we get frustrated with breaking changes made to the source code over which we have no control.  This is an issue when integrating with any third party but it seems particularly true of Indy between simple point releases.  kbmMW is only using a relatively small set of Indy APIs but we get repeatedly asked how to get the framework running with Indy version X, or Y, or the latest development snapshot etc.  We have transports based on Synapse and have never needed to update these.
  • There were requests being made for us to supply messaging transports based on Synapse, motivated partly by the above issues with Indy.
  • The current transports have scalability limits.
 

Are you saying kbmMW is not scalable?

I should expand on the last point.  kbmMW is a hugely scalable framework.  I am not implying anything to the contrary with the last statement.  A single server will scale up to many hundreds of users without problem.  This is great for most applications that use the traditional request response operational model – your typical CMS application etc. 

 

When we introduced messaging the framework took on a paradigm shift.  To people building standard applications this will go largely unnoticed but internally the framework behaviour has changed to work in terms of discrete messages.  When applications are built using the messaging transports developers can still call services just like they did before.  But the way these calls are expressed is different.  A request is simply a message sent from a client with the first part of its subject being REQ.  REQ stands for request funnily enough.  The framework takes this message and decodes it into a call to a certain service/method pair and processes the request.  Results from the request are sent back to the caller using another message with the subject RES.  RES stands for response.  I’m digressing a little but the point is people can use the messaging transports without losing any existing functionality.  In fact it opens up huge possibilities when we add intelligent routing to these REQ messages.  Messaging is incredibly powerful!

 

What else can people gain from using messaging then?  

To use a cliché, access to a whole new world of applications where discrete message passing is the key.  I have been involved in a project requiring the distribution of real-time information to huge numbers of clients or potential clients.  Majority of the time the clients may be doing nothing other than monitoring some statistical feeds.  The amount of data they receive is small but there are a lot of them.  By this I’m talking about potentially millions.  The question is how to support millions of client connections to the kbmMW messaging based WIB?

 

It is time for another digression – why can’t we use the Indy messaging transport?  To answer this question I need to explain how the Indy, Synapse and DXSocks TCPIP based transports work.  The focus of this is the server or hub in messaging terms.

 

Each client connects to the hub using a TCPIP socket.  Data is sent from the client and transferred by the TCPIP stack over to the server where it is presented as a memory buffer to be read.  But how do we know there is data ready for us to read?  Simple – we call various APIs to tell us if there is a buffer waiting for us.  That is the problem.  Read it again, “We call various APIs to tell us if there is a buffer waiting for us”.  We have to poll the status of the socket.  To do this the usual arrangement is to have a thread executing in a loop polling the socket.

 

Here is the code pattern for the server transports.

 

While the client is connected

            Test socket for data

            If data is present

                        Process data

            Else

                        Sleep for a moment 

In the Indy transport their source implements the loop.  For Synapse, which is API based, we implement it in ourselves.

 

A quick quiz!

 

Q. How many threads does it take to service 100 clients?

A. 101  (There is also a separate thread responsible for listening for new connections and creating the new socket/thread pair). 

 

Q. How many threads does it take to service 500 clients?

A. 501

 

Q. How many threads does it take to service 10000 clients?

A. Too many!

 

As more and more threads are created there is more and more contention for CPU resource.  This takes the form of expensive thread context switches and there comes a point where the majority of CPU time is spent simply switching from thread to thread.  The official description is thread thrashing. Throughput of the server decays rapidly.

 

How do we avoid thread thrashing?

The answer is relatively obvious – use less threads!  One approach that can be used is to have each thread handling more than one socket but this is still essentially a polling solution.  We need something else.

 

Welcome to IO Completion Ports

This was introduced in Windows NT 3.5 and is a complete paradigm shift.  Instead of polling sockets for data we instead get the TCPIP stack to notify us when new data has arrived.  It sounds obvious and simple.  Of course it doesn’t work out to be quite that simple. The goal in any server design should be to try and incur as few context switches as possible by avoiding threads being unnecessarily blocked whilst at the same time maximising parallelism using multiple threads.  The ideal situation is to have a thread actively servicing a client request on every processor and for those threads not to block in the event that additional requests are waiting when they complete their current request.  For this to work there must be a way for the application to activate another thread when others are busy processing another I/O operation.  Windows NT 3.5 introduced a set of APIs based on something called the completion port.  Applications can associate a completion port with a TCPIP socket.  Data coming in off the socket results in a completion packet being queued to the port for processing.  A set of threads associated with the port read off the data packet and process it.  When a completion port is created we specify a concurrency value.  This is the maximum number of threads that can be actively processing data packets queued to the port at any one time.  The Windows Kernel performs the creation and management of these threads.  The aim is to have one thread active at any given time per processor.  A typical rule of thumb for the concurrency value is 2 times the number of CPU cores.  At this time we have an open mind on this until we perform some more scalability tests.  The Windows scheduler attempts to reduce context switches by selecting the same thread to process request N + 1 after processing request N this allows CPUs to be utilised to near their full capacity.

 

 

Running the quick quiz again!

 

For a 4 CPU server 

Q. How many IO worker threads does it take to service 100 clients?

A. 8 using rule of thumb  

Q. How many threads does it take to service 500 clients?

A. 8 using rule of thumb  

Q. How many threads does it take to service 10000 clients?

A. 8 using rule of thumb  

What a result!

It’s not just about threads.

We have a mechanism to reduce the thread count and avoid thread thrashing by using our completion port.  But there is another effect we can overcome by having our own socket layer.

 

As competent and complete as they are Indy and Synapse are complete communication stacks.  By this I mean they present a nice uniform view of the data coming off the socket.  When we detect in our listener thread that there is data on the socket we are presented with the entire stream of data sent over from the client.  The buffer size can range from a few bytes to multi-megabytes. kbmMW must then take a copy of that buffer in order to safely work with it.  That is a copy operation.

 

IOCP works slightly differently.  It is not just the IO worker threads that are associated with the completion port; the socket is also using a special data structure of type WSAOverlapped.  We can see from the diagram above that somehow data passes through the completion port from the TCPIP stack and gets processed by the IO worker thread.  The data passes through by being copied to memory buffers that we supply.  We supply the completion port with our memory buffers by posting them to the completion port using the WSAOverlapped structure associated with the socket.  What this means is that the IO worker threads can copy data into our buffers straight off the TCPIP stack without needing a context switch.  If we supply enough buffers to the completion port before the client request comes in, then the whole request can be copied to buffers without a context switch.  In addition if we’re processing a request in the framework and another request (read message actually) comes in from the client on that socket another IO thread can start processing it in parallel. 

 

What does this mean?  It means that we are removing the large buffer copy of the complete stream that we have to perform for Indy/Synapse.  This will boost performance.  In addition if we manage the number of buffer we post to the completion port well we can allow the IO worker thread to pull data off the TCPIP stack unhindered.  It is worth remembering that the IO worker threads are now running in kernel mode not user mode.   In essence – kbmMW is moving closer to the TCPIP stack with the removal of Indy/Synapse.

 

I’ve been writing for a while now and need to get back to some coding.  There will be some further blog entries on IOCP because there is a lot more to talk about.

Keep well

Richard

 

 

kbmMW Spider 2 for dotnet

by Richard 19. October 2007

 

The demos are built.  The help file is writen.  The installer is done. 

Introducing kbmMW Spider 2 for dotnet Cool

Firstly a big thanks for the field testers.  They made the product better and proven in a live environment including our own myc4d.com and turbomiddleware.com sites.

I thought I'd run through the shipping demos,  they're kind of fun.

There is a W32 appilication server based on the standard BDE demo we ship with the framework.  I decided to implement the classic biolife demo. 

Here is a code sample form it's main service

function TTestQuery.ProcessRequest(const Func: string;
  const ClientIdent: TkbmMWClientIdentity;
  const Args: array of Variant): Variant;
begin
  if (Func='FETCHBIOLIFE') then
    StreamBiolife else
  if (Func='FETCHBIOLIFEOBJECTS') then
    StreamBiolifeObjects
  else
    inherited ProcessRequest(Func,ClientIdent,Args);

end;

procedure TTestQuery.StreamBiolife;
begin
  self.BIOLIFE.Open;
  try
    self.BIOLIFE.SaveToStreamViaFormat(self.ResultStream,self.kbmMWBinaryStreamFormat1);
  finally
    self.BIOLIFE.Close;
  end;
end;

There are two service methods supported.  The first one simply opens BioLife.db and streams the resultset down the wire.

I have three demo clients.  They are

  • Winforms client built with VS2005
  • VCL.net client built with D2007 for dotnet
  • ASP.net client built with VS2005

 

Here is the important code for the Winforms sample

        private void button1_Click(object sender, EventArgs e)
        {
            C4D.kbmMW.Transports.TCPIPTransport transport = new C4D.kbmMW.Transports.TCPIPTransport();
            transport.Port = 3000;
            transport.Host = "127.0.0.1";

            transport.Connect();

            C4D.kbmMW.Client.SimpleClient client = new C4D.kbmMW.Client.SimpleClient(transport);


            C4D.kbmMW.Data.DataAdapter da = new C4D.kbmMW.Data.DataAdapter();
            da.FormatterType = C4D.Common.Formatters.FormatterType.kbmMW;

            client.SendRequest("KBMMW_QUERY", "", "FETCHBIOLIFE");

            System.Data.DataTable dataTable = new DataTable("biolife");

            da.FillDatatable(dataTable, client.ResultStream);

            dataGrid.DataSource = dataTable;
            dataGrid.Columns["Graphic"].AutoSizeMode = DataGridViewAutoSizeColumnMode.AllCells;
        }

And here is the output

 

Cool

Here is the VCL.net code sample

procedure TForm1.Button1Click(Sender: TObject);
var
  transport: C4D.kbmMW.Transports.TCPIPTransport;
  client: C4D.kbmMW.Client.SimpleClient;
  da: C4D.kbmMW.Data.DataAdapter;
  dataTable: System.Data.DataTable;
  ADOBridge: Borland.Vcl.ADONETDb.TADONETConnector;
begin
  transport:= C4D.kbmMW.Transports.TCPIPTransport.Create;
  transport.Port:= 3000;
  transport.Host:= '127.0.0.1';

  transport.Connect();

  client:= C4D.kbmMW.Client.SimpleClient.Create(transport);

  da:= C4D.kbmMW.Data.DataAdapter.Create;
  da.FormatterType:= C4D.Common.Formatters.FormatterType.kbmMW;

  client.SendRequest('KBMMW_QUERY', '', 'FETCHBIOLIFE');

  dataTable:= System.Data.DataTable.Create('biolife');

  da.FillDatatable(dataTable, client.ResultStream);

  ADOBridge:= Borland.Vcl.ADONETDb.TADONETConnector.Create(nil);
  ADOBridge.DataTable:= dataTable;

  self.DataSource.DataSet:= ADOBridge;
  self.DBGrid1.DataSource:= self.DataSource;

end;

and the output 

 

 

The Delphi grid doesn't recognise the images - but owner draw would cope with that.

Here's the ASP.net VS2005 sample.

    protected void Page_Load(object sender, EventArgs e)
    {
        //  We're using a connection pool for this sample, see Global.cs
        C4D.kbmMW.Client.PooledSimpleClient client = new C4D.kbmMW.Client.PooledSimpleClient(Global.ConnectionPool);

        C4D.kbmMW.Data.DataAdapter da = new C4D.kbmMW.Data.DataAdapter();
        da.FormatterType = C4D.Common.Formatters.FormatterType.kbmMW;

        client.SendRequest("KBMMW_QUERY", "", "FETCHBIOLIFE");

        System.Data.DataTable dataTable = new DataTable("biolife");

        da.FillDatatable(dataTable, client.ResultStream);

        this.GridView1.DataSource = dataTable;
        this.GridView1.DataBind();

    }

You will notice the use of Global.ConnectionPool

This is the connection pooling feature of Spider2 and is setup like this in Global.cs

public partial class Global : System.Web.HttpApplication
{
    private static C4D.kbmMW.Client.ClientConnectionPool connectionPool;
    public Global()
    {
        connectionPool = new C4D.kbmMW.Client.ClientConnectionPool();
        C4D.kbmMW.Transports.TCPIPTransport transport = new C4D.kbmMW.Transports.TCPIPTransport();
        transport.Host = "127.0.0.1";
        transport.Port = 3000;
        connectionPool.Template = transport;
    }

    public static C4D.kbmMW.Client.ClientConnectionPool ConnectionPool
    {
        get { return connectionPool; }
    }

    protected void Application_Start(Object sender, EventArgs e)
    {
        connectionPool.Active = true;
    }
 }

 

 

and the output

Finally, the DataAdapters use the VCL streaming support.  There is nothing to stop us using this directly.

The second method on the server was

procedure TTestQuery.StreamBiolifeObjects;
var
  writer: TWriter;
begin
  writer:= TWriter.Create(self.ResultStream,1024);
  self.BIOLIFE.Open;
  try
    writer.WriteInteger(self.BIOLIFE.RecordCount);
    while not self.BIOLIFE.Eof do
    begin
      writer.WriteInteger(self.BIOLIFE.FieldByName('Species No').AsInteger);
      writer.WriteString(self.BIOLIFE.FieldByName('Category').AsString);
      writer.WriteString(self.BIOLIFE.FieldByName('Common_Name').AsString);
      writer.WriteString(self.BIOLIFE.FieldByName('Species Name').AsString);
      writer.WriteFloat(self.BIOLIFE.FieldByName('Length (cm)').AsFloat);
      writer.WriteFloat(self.BIOLIFE.FieldByName('Length_In').AsFloat);
      writer.WriteString(self.BIOLIFE.FieldByName('Notes').AsString);
      self.BIOLIFE.Next;
    end;    // while
  finally
    writer.Free;
    self.BIOLIFE.Close;
  end;
end;

Here we use a TWriter component to write out the Biolife data directly onto the resultstream.

On the ASP.net client we have this code to decode it.

    class BiolifeObject
    {
        private int speciesNo;
        private string category;
        private string commonName;
        private string speciesName;
        private double lengthCM;
        private double lengthIn;
        private string notes;

        public int SpeciesNo
        {
            get { return this.speciesNo; }
            set { this.speciesNo = value; }
        }
        public string Category
        {
            get { return category; }
            set { category = value; }
        }
        public string CommonName
        {
            get { return commonName; }
            set { commonName = value; }
        }
        public string SpeciesName
        {
            get { return speciesName; }
            set { speciesName = value; }
        }
        public double LengthCM
        {
            get { return lengthCM; }
            set { lengthCM = value; }
        }
        public double LengthIn
        {
            get { return lengthIn; }
            set { lengthIn = value; }
        }
        public string Notes
        {
            get { return notes; }
            set { notes = value; }
        }
    }

This is a class to accept the data and we bind it like this.

    protected void Button1_Click(object sender, EventArgs e)
    {
        //  We're using a connection pool for this sample, see Global.cs
        C4D.kbmMW.Client.PooledSimpleClient client = new C4D.kbmMW.Client.PooledSimpleClient(Global.ConnectionPool);
        client.SendRequest("KBMMW_QUERY", "", "FETCHBIOLIFEOBJECTS");

        C4D.Common.VCL.VCLReader reader = new C4D.Common.VCL.VCLReader(client.ResultStream);
        System.Collections.ArrayList objects = new System.Collections.ArrayList();

        int ObjectCount = reader.ReadInteger();

        while (ObjectCount > 0)
        {
            BiolifeObject bo = new BiolifeObject();
            bo.SpeciesNo = reader.ReadInteger();
            bo.Category = reader.ReadString();
            bo.CommonName = reader.ReadString();
            bo.SpeciesName = reader.ReadString();
            bo.LengthCM = reader.ReadFloat();
            bo.LengthIn = reader.ReadFloat();
            bo.Notes = reader.ReadString();
            objects.Add(bo);
            ObjectCount--;
        }

        this.GridView1.DataSource = objects;
        this.GridView1.DataBind();

    }

We marshall the streamed data off the resultstream using a VCLReader and create an arraylist of Biolife objects.  These can be bound to directly from the grid. 

The output is as follows (hit FetchAsObjects), although the property ordering isn't maintained.

There you go and blindingly quick.

Look out for the release announcement.  It will be available at MyC4D.com

Richard

Native Delphi kbmMW client built with Highlander

by Richard 4. September 2007

Components4Developers have been given special permission by CodeGear to talk about the Highlander field test release.

In this third blog entry about the CodeGear Highlander pre-release I'm going to demonstrate a full kbmMW native Delphi client in operation.

It is going to call a windows 32bit kbmMW server that has a service supporting three operations for this demo.

1. A simple echo remote procedure call

2. Streaming a TkbmMemtable down the wire to the client

3. Execution of a named query that returns a list of C4D products form a SQL Server 2005 database

Below is the user control I designed in Highlander for the three demos

1. A simple echo remote procedure call

Here is the code for the Ping button

To make things simpler I encapsulated the wiring up of the TCP/IP transport into a connector class to declutter these samples.  This simple demo sends the content of the text box and sends it to our service named KBMMW_QUERY for processing my the method PingServerPingServer simply returns the text sent to it as argument 0.  We take the return value and update a lable to the right of out ping button.

Here's the result

 

2. Streaming a TkbmMemtable down the wire to the client

The server being used for these demos is the same one used when developing Spider 2 - our extended C# client.  On our service we have a method called GemMemtable that creates and defines a TkbmMemtable in the fly and populates it with two records.  The whole memtable structure and data is then formatted onto the resultstream and sent down the wire.  At the client we must do the reverse.

For this sample we perform the decode into a local TkbmMemtable and then bind that to a grid to visualise the result.  Here is the code

 

Once again we call our service but this time we execute the method GetMemtable. After the call the data from our server resides in the stream client.resultstream.  We create a formatter, in this case the kbmMW formatter, and populate a TkbmMemtable using it.  This can be seen in the line memtable.LoadFromStreamViaFormat(client.ResultStream,formatter)

After this point we have our TkbmMemtable correctly structured and populated with data from the server.  Because of the dotnet extensions I talked about in blog 2 I can bind this directly to the standard ASP.net Gridview control.

Here is the result:

3. Execution of a named query that returns a list of C4D products form a SQL Server 2005 database

kbmMW users will be familiar with the concept of named queries.  We want to promote good n-tier practices where business knowledge is encapsulated at the server and not the client.  Named queries promote this by allowing clients to execute a query by name without needing any knowledge of the SQL that will execute.

Here is the code:

In this sample we create a TkbmMWClientQuery to accept the data from our server.  We bind it to a formatter, in this case the TkbmMWBinaryStreamFormat, and set it's Query text to be our named query @highlanderDemo.  Next we simply call Open and the rest is automatic.  Once more we bind the result to a gridview.  The named query that our client had no knowledge about lists the products available on myc4d.com.  Here is the result:

One final note to end with and this is mainly of interest to kbmMW users.  During the very early phase of the Highlander FT there was no Indy 10 assembly to use.  In order to progress we wrote our own native Delphi socket client.  Here is a snippet of the client connector encapsulation used for these demos where you will notice the new transport TkbmMWTCPIPDotNetClientTransport Cool:

A final final note.  Highlander has been great in terms of code compatibility.  Almost all the codebase has ported without much effort. Most changes have been due to alterations in the dotnet framework itself.  A good sign.

Richard

kbmMemtable with dotnet extensions in Highlander

by Richard 2. September 2007

 

Components4Developers have been given special permission by CodeGear to talk about the Highlander field test release.

A project that I have running which has slipped on our release schedule is kbmMemtable dotnet extensions.  This uses the standard TkbmMemtable with some special plug-in extensions.  The extensions are relevent to dotnet only and extend the memtable to be fully bindable in dotnet.  I had this running under dotnet 1.1 built using BDS2006 but wanted to take it further by re-optimsing some of the memtable internal structures to boost it's speed and capabilities.  Unfortunately at the time the internal restructuring I had in mind used features only available in dotnet 2.0 hence the slip. 

Now that Highlander has come along I've re-visited my project. After rebuilding my assembly with the plugin extensions I created a new ASP.net project and added a reference to it - kbmMemProD2007dnRun.dll in this case.

 

I then added some code into the page_load event of the default WebForm1.aspx where I create a kbmMemtable and add three fielddefs to it.  These are of type String, Integer and Float.

I then Open the table and add 20 records to it.  My page contains a DataGrid and once I've added my records to the memtable I bind it to the datagrid.

 

Here is a screenshot of the running application:

So - some might wonder what the big deal is?

Well there are a few things going on. 

1) kbmMemtable is running under dotnet 2

2) we're able to bind kbmMemtable as a datasource to standard dotnet controls

3) kbmMemtable is still a TDataset descendent and hence can be bound to any VCL.net control too

4) kbmMW queries are descended from TkbmCustomMemtable - hence kbmMW resultsets also support 1, 2 and 3 above.

In combination with Spider 2 that I blogged about last time I hope you can see it is now possible to pass around and work with kbmMemtables on dotnet 1.1, 2.0 and 3.0, win32 and linux.  More to the point - you can choose to visualise the data using any combination of VCL, VCL.net, WinForms or ASP.net controls/components. 

My next blog will show kbmMW working in Highlander. 

Richard

kbmMWSpider Extensions and Highlander

by Richard 26. August 2007

First the disclaimer

Components4Developers have been given specific permission by CodeGear to discuss Highlander in public.

All screenshots are from a pre-release beta version of Highlander.

I decided to run up the kbmMWSpider extensions for dotnet in Highlander and check that it all worked nicely.  It did and it was great to see Spider and Delphi code back together for DotNet2.

Here is a sequence of screenshots to show you what I did but basically I ran up an ASP.net application, added a page and put a grid and a button on it.  In the OnClick event of the button I make a call out to a kbmMW server and bind the returned data to the grid.

Here we go

1) Startup the Highlander IDE

2) Add the kbmMWSpider20.dll assembly to the references section of the project

3) Drop a GridView and Button control onto the Default.aspx page

4) Add the appropriate units to the implementation section of the page.

5) Implement the code for the button.

6) Run the application!

  

When I first ran the project up I hit a problem because I'd previously changed the application server return values.  However in debugging my ASP.net client the seemless debugging in Highlander was nicely highlighted.  Here is a screen shot of some C# code of kbmMWSpider that I traced into seemlessly.

 

There we have it.  kbmMWSpider working nicely in Highlander.

I'm planning another Blog showing the full client framework running but for now it's off to get some breakfast Smile 

Richard

What exactly is corporate software?

by Kim 23. July 2007

We had on our newsgroups at news://news.components4developers.com recently a question about if kbmMW can be considered a 'corporate product'.

My answer to that was that in our opinion, yes, but we certainly dont have the marketing muscles of the big ones that are considered 'corporate'. I consider something to be 'Corporate' if the following conditions are fullfilled:

  1. It serves a corporate purpose
  2. It provides a reasonable ROI
  3. It is backed by a stable organization/vendor
  4. It has a future
  5. It is maintainable
  6. It is maintained

Lets take them one at a time:

1) Well.. if it doesnt serve any purpose in corporate sense (eg. a company cant use it for something useful), it isnt corporate.
2) If it is costing an arm and a leg to install or use to serve purpose 1, then it isnt corporate.
3) If its looking like abandonware, or anyway highriscware, then it isnt corporate. A company has to be able to trust that the vendor is around to support it.
4) If the product do not have a future, its not corporate. There is no idea for a company to spend energy on something without a future.
5) If the product is next to impossible to maintain and require too many manhours, its not corporate.
6) If the product is not maintained it means that the product is reaching a status that would kick in point 4. No future. Its not corporate.

In reality I see large corporations use products that they consider corporate, but that typically do not live up to those 6 criterias. Lets look at some examples:

Sun - J2EE: For the most part it doesnt live up to the 2nd and 5th criteria. It is typically difficult to get to perform (why there are lots of highly paid contractors in that area), and its equally difficult to deploy an application server. Some vendors of application servers have made it somewhat easier than others, but the base line is that a scalable, performing production deployment is not easily made.

Sun - JMS: In addition to the difficulties in deploying a production system, thus violating point 5, it also violates 2 as JMS by itself does absolutely nothing. Routing mechanisms, request/response mechanisms, quality of service mechanisms etc. must all be handcoded by each project wanting to implement JMS. There exists frameworks based on JMS that makes this somewhat easier, but that may actually end up in making the maintainability even worse.

Microsoft - VB: The number of VB3-6 applications still running in corporate environments are quite staggering. However it is abandonware, and no longer maintained by MS, hence violating 3, 4 and 6. In addition upgrading in the good old days from one VB version to another usually required rewriting specially the database access code as MS had an idea to reinvent the wheel on a 6 months basis.

Microsoft - .Net: That surely must be corporate, right? Well.. the corporations that based millions of US$ on VB in the good old days should actually have removed the 'Corporate' label of that product, although most havnt. The trackrecord (as in whats the chance your application can be recompiled with the next version) of MS development products except perhaps for Visual C/C++, has not really been something to make corporations feel a good warm fuzzy gut feeling. The amount of US$ spend to rewrite existing applications to move them to the new version or to support the new Vista etc. is simply staggering. People who believed in WinForms being the future also have to count with uncertainty to what amounts of rewrites are needed.

IBM - Visual Age for Java. It was actually a very capable Java development IDE, with some features that made it really unique. After IBM chose to drop VA for Eclipse, developers using VA had no choise but to follow to the new platform, despite that the new IDE didnt contain all the wizard functionality originally found in VA.

IBM - ComponentBroker. IBM's CORBA solution for Java and C++. It was dropped about 4 months after I attended a quite expensive course in the product. Fortunately we the company I worked for at the time didnt get to actually integrate it into something, but if we had, it would have cost serious US$ to replace it with something still maintained and supported. In addition it was cumbersome and errorprone to use it. However it was still considered 'Corporate' all the way until it was dismissed.

In my opinion, the label 'Corporate product' often is a synonym with big, difficult to use, difficult to manage, difficult to deploy, overly expensive software backed by the 3-4 most PR active IT companies.

What is your definition of 'Corporate'?

Kim

Focusing on the details - kbmDiff

by Kim 4. July 2007

People who have registered on our myc4d portal or on TurboMiddleware now have access to a new cool freeware utility - kbmDiff.

As the name says, its a tool that can compare files and show you the differences. In addition it allows you to manually merge selected differences from one file to the other.

I cant take the credit for all the work behind it, as 95% of it was made by Angus Johnson as a demo for his TDiff component. I have however added several features to it like:

  • Ability to compare normally non comparable files like unformatted XML (ofcourse using kbmMW's DOM XML parser/generator) and binary files.
  • Scrollwheel support
  • Support for clicking and dragging in the right differences summary pane to show where the difference is in the text

and some more details here and there to imo improve the user experience.

The application is actually quite versatile, and can easily be compared with the typical commercial tools in the market.

Angus Johnson, who deserves every credit, have a home page here: http://angusj.com/delphi/ where the original source code for TDiff and the TextDiff (as his project was called) can be found.

To get the kbmDiff version of it signup at: www.myc4d.com or at www.turbomiddleware.com

best regards

Kim Madsen

Powered by BlogEngine.NET 1.0.0.0
Theme by Components4Developers

Calendar

<<  November 2008  >>
MoTuWeThFrSaSu
272829303112
3456789
10111213141516
17181920212223
24252627282930
1234567

View posts in large calendar

Categories


Archive

Blogroll

Disclaimer

The opinions expressed herein are personal opinions and do not necessarely represent Components4Developers view in anyway. Any forward looking statements are not a guarantee of future direction and Components4Developers retains the full right to alter our plans in any way.

© Copyright 2008

Sign in