---------------------------------------------------------
Brosco's Liberty Basic Newsletter - Issue #13 - July 98
---------------------------------------------------------

Aaaaah, number 13 - unlucky for some - lucky for others.  If you 
don't like to receive large emails with attachments to clutter up
your inbox - then I'm sorry - issue #13 is definitely not going
to make you happy.  While I don't expect to produce many issues
as large as this one, it will happen occasionally.  Please let
me know if this causes you a problem - afterall - this is YOUR
newsletter - so PLEASE let me know your thoughts about this.

        In this Issue:
            1)  Hey Carl - we need R2.00, 32 bit and a compiler!
            2)  Using Secondary Indexes
            3)  Data Dictionary - DB Application Generator.

	Attachments:
             NL13.zip  Includes the following files:
            1) Update4.bas, video5.dat, video5.idx, video5.idx2
               The programs and datafiles to demonstrate 
               "Using Secondary Indexes". 
            2) DBdll.dll Indexing DLL - now works on some 486's
            3) DBdll.doc Documentation (in Word format) 
            4) dbcalls.lb, dbspeed.lb dbdemo.lb 
               Demostration programs for DBdll.
            5) DD4.bas, dd.txt, fields.txt, ctl3dv2.dll
               DB Application Generator program.

         Acknowledgements:

            DBdll.dll is the result of the combined efforts of
            Brosco and Dean Hodgson.  Dean provided most of the
            ideas to make DBdll more than just 'yet another'
            indexing program.  Dean also performed much of the
            Beta testing and produced all the documentation
            (included in this zip).  A very special thanks for
            all the assistance, ideas and encouragement.

            Of course, Tom and Alyce Watson also carried out
            a lot of the Beta Testing - especially with the
            Application Generator.  Tom is a very thorough
            beta tester who manages to 'test to destruction'
            the programs provided here.  His efforts, on this
            (and many other Newsletter programs) has minimised
            the possibility of me shipping sample programs with
            bugs.  Thank you Tom and Alyce.
---------------------------------------------------------
1)  Hey Carl - we need R2.00, 32 bit and a compiler!

There hardly seems to be a day going by without someone asking
about one of the above.  With the day approaching when Anthony's
compiler, and Carl's R2.00 will be available, I thought that it
was time to put some important issues into perspective. 

               What will they do for us?

From time to time Carl gives us a hint of some of the things 
to come in Release 2.00 - and yes - I'm looking forward to it.
I'm looking forward to being able to have 'true' basic 
Functions;  also the ability to process Window's Callbacks.

And with any new release, there're always a few extra goodies
to make life a little easier - but don't think that R2.00
will make your programs bug-free.  YOU still need to debug
your own code. 

Dont think that your programs are going to run at twice 
the speed - they won't!

Ah hah - thats where 32 bit comes in - or better still a
compiler - that will make every thing happen in a flash!

Sorry - wrong, wrong, wrong!

I dont want to sound discouraging - but these things will
only make a small difference to your overall application.
True, in certain situations, they can make a big difference -
but generally no!

Most Window's programs are designed to accept some input
from a user - process that input - and then display the
results.  Even a games program works in this mode.  There
are very few programs that must process a large amount of data
between receiving input and displaying the results. 

If you have written applications that are running too slow 
at the moment - don't blame Liberty Basic.  You need to 
take a good hard look at your code.  In programming there
are 'many ways to skin a cat.'  You need to isolate the
areas that are the bottlenecks - and find alternative ways
to do the same things.  Something that takes only a few
microseconds of extra CPU time - when put in a FOR...Next
loop 1000 times - means that it takes a few extra seconds.
And sitting in front of a computer screen, when nothing is
visibly happening, a few seconds is a looooong time.

Let me give you some examples.  The first one that truly
dominates my mind is Michael Rankin's SpellChecker program.


Case Study 1:

Michael had this program working long before it was posted
for the group to see.  It was working - but performance was
unacceptable.  He wrote to me to tell me what he was trying
to achieve, and asked if he was being too ambitious.  We 
isolated the problem areas and discussed several approaches 
and alternatives.  Eventually, Michael hit upon the idea of
using Binary Search techniques (see my online tutorial for
array searching techniques) against his RANDOM file.  To
be honest - I told him that I didn't think it would help -
BUT - I still said that he should try it.  And the results
speak for themselves!  A spell checker written 100% in LB,
and not a hint of a performance problem!
(This is the Spell Checker that's used in the LB NotePad 
exercise in a previous newsletter).

 
Case Study 2:

Alyce posted a program to View, Cut and Paste Windows API
calls to LB programs.  But the problem was, that as the
API reference list grew (now over 120Kb of text) it was
taking increasingly longer to load and find the Call to be
displayed.  There were two problems:

1)  The size of the file, and
2)  LB's INSTR function.

It appears that the performance of INSTR suffers considerably
once the string being searched is over 32KB.  Even finding
something in the first position of the string takes 10 times
as long.  One approach would be be to blame Liberty Basic -
say there's nothing that can be done - and forget it.  BUT,
a better approach is to to find alternatives.

At my suggestion, the file was split into 5 files, each less 
than 32KB.  This solved 2 problems, load time was now quick - 
and the LB INSTR search on strings less than 32KB is superfast.  
So all that was left to solve was to know which of the 5 files 
to load.  Since Alyce already had a separate file with all the 
functions listed, she simply put a number (1 to 5) next to the 
function to state which file it was in - problem solved!


Case Study 3

Accessing data in Random files is always a problem - in any
programming language.  And having a 32 bit LB or a Compiled LB would 
not make a lot of difference.  The reason being, searching 
through a Random file is influenced more by the speed of
your disk drive, than it is by the speed of the CPU.  This 
is why I wrote the DBDll.  It overcomes problems here that
a compiled version of LB can't.  Even if you could write
DBdll in Basic, it is preferable to have functions like these
in a DLL so that they don't 'bloat' the size of every program
that uses them.
  

SUMMARY:

LB Release 2.00
Yes, I'm looking forward to it and the extra functions that will 
come with it.  But since I still haven't leaned how to use the 
full capabilities of LB V1.4, it is not a pressing matter.

LB 32 - Bit
I don't even know if one is planned, and to my way of thinking,
we dont need one.  Sure, it would run slightly faster than the
16 bit version, BUT we would no longer have the compatibility
to run on 16 bit.  (When developers create a 32 bit version of
an existing program - there is a big temptation to take advantage
of some of the extra features available - the two versions soon
have differences).  LB now runs on both platforms - lets keep it
that way.

LB Compiler.
This would give us: faster program load and faster run time, 
especially if we have used good coding techniques to begin with.  
Although TKN's are fairly small files to use for distribution to 
other LB'ers, the compiled files would be a definite impovement 
over distributing the runtimes to the general public.

The LB Compiler will benefit many people.  But the majority of 
the readers of this newsletter are in the category of 
'self-taught hobbiests'.  Most programs are written for your 
own use and distribution to the LB community, and these generally 
don't need to be compiled.

CONCLUSIONS:

The point I'm trying to make here is:  DON'T blame 16 bit
LB for the performance problems of your program.  It is
probably because YOU are using the wrong techniques to
achieve your goal.  And getting a faster version of bad
techniques - still doesn't make them good techniques!


Another point that I must make here:   There is absolutely, no
way that you can become a self-taught hobbiest C++ programmer
(or Visual Basic, or Delphi, or....... etc).  Liberty Basic 
is disigned in such a way as to remove many of the difficulties
involved in learning Windows programming.  This does lead to some 
overheads - but what would you really prefer - a program that
runs a few milliseconds slower, but can be created in a few hours - 
or one that is a few milliseconds faster - but takes you days 
and days to produce?


SPEEDING UP YOUR PROGRAMS
If you have a program that is currently performing too slowly,
let me have a look at it.  BUT, don't send me your code.  Isolate
the area that needs improvement and post it to my message board.
That way, everybody can look at the problem, offer suggestions 
and see the final solution.

IF you have a problem where it would be impractical to post it
to the MsgBoard (because of size, support files, etc), OK, send 
it to me.  But I reserve the right to post the problem and the
solution to the message board.  Also, problems posted to the 
Board will receive priority.

To access my message board - go to:

http://users.orac.net.au/~brosco


A bit of background on Brosco!
I have been a professional programmer for over 30 years. I
have used over a dozen different programming languages,
including Assembler, Pascal, and several you will not have heard
about (mainly because most of my programming was on Mainframes,
not PCs).  I have always had a MicroComputer, ever since they
first became available - and so programming these has been my
hobby.

People who know my background often ask why, with my experience,
do I elect to use Liberty Basic for programming?

1)  I had never programmed in the Windows environment.  I tried
a copy of Visual Basic and also Turbo Pascal for Windows.  But
I just couldn't seem to grasp the concept of Object Oriented
Programming.

2) I discovered a shareware version of Liberty Basic on a CDROM
on the cover of a PC Magazine.  I decided to give Windows 
programming another go.  I couldn't believe it.  I produced a 
small, but useful utility program the same day!!

3)  The knowledge that I have gained through using LB would
probably allow me to learn programming in VB (or some other
language) if I needed to.  But why bother.  LB can achieve 
most of the things I need - provide acceptable performance -
plus I can write a LB program in a fraction of the time it
would take in another language. 


Carl has done a fantastic job of taking the mystery out of
Windows programming and making it available to the masses.
There isn't a single other product that comes close to
providing power and functionality - yet remaining simple
to use.  My sincere thanks to Carl for producing such a
remarkable product.

---------------------------------------------------------
2)  Using Secondary Indexes


Well, I hope that the last issue concering Random files didnt 
confuse you too much, because this issue is going to 'knock 
your socks off'!!!!

Your saw last issue how we can index a file, plus we used a
concept called 'unique keys'.  That is, we only allowed one
entry for each Primary Key of Cassette number.

This time I will introduce the concept of Secondary Indexes
and the idea of duplicate keys. 

Suppose that apart from browsing through our database, or
just typing in a Cassette number to see what's on it - how 
about if we gave the capability of searching the Database
for our favourite movies stars.  Now, in your collection,
you would probably have several movies with your favourite
star - so we will need to allow several entries with the 
same key.  This means that the index must allow DUPLICATE
keys.

        calldll #db, "CreateIndex", _
            fn2$ as ptr,_     ' Full path to the index file
            24 as word, _     ' Length of the key - maximum 256 bytes
            1 as word, _     ' 0 = No duplicates , 1=Duplicates OK
            result as word   ' 0 = OK, anything else is a file error

OK - this defines our second index - we have told the DBdll that
the key will be 24 bytes (the actors name) and that Duplicate 
Keys will be allowed.

    calldll #db, "OpenIndex", _
        fn2$ as ptr, _      ' File name used in CreateIndex
        0 as word, _       ' Share mode
        0 as word, _       ' Normal operation
        hx2 as word         ' Handle of the Index file to be
                           ' used in subsequent calls

We also need to OpenIndex for this new index as well - note that
we have a different handle - hx2 - for this index.  This is how 
we distinguish between the two indexes when we are communicating 
with DBdll.  


ADDING NEW RECORDS  

When we added a new record to the database, we needed to check
that this key wasn't already in use by another record.  When we
allow 'duplicate' keys - we don't need to worry about this - we
just go ahead and Add it.

    key2$ = trim$(mainActor$) + chr$(0)
    CallDll #db, "AddKey", _
        hx2 as word, key2$ as ptr, cNum as long, result as word


DELETING RECORDS WITH DUPLICATE KEYS

When we Delete a record - apart from deleting the main Index key,
we also must delete the Secondary Index key - otherwise we would 
be left with a key pointing to a deleted record:

    key2$ = trim$(mainActor$) + chr$(0)
    CallDll #db, "DeleteKey", _
        hx2 as word, key2$ as ptr, cNum as long, result as word
        
Note that when we delete a key where Duplicates are allowed, we
must also tell it which RecordNumber is the associated with the
key - otherwise it wouldn't know which one of the duplicate keys
to delete!


UPDATING RECORDS 

If we update a record - and the Key changes - we must delete 
the old key and Add the new key - just the way we did for the
Primary Index.  You can see the code for this in the program.


SEARCHING USING THE SECONDARY INDEX.

To make use of the Secondary Index, I could have just used a 
PREVIOUS, NEXT and a READ KEY function - in exactly the same 
way that we used them with the Primary key - but I think its 
time to show you some really sexy stuff.  

Now dont get confused here.  We are going to use the Secondary 
Key in a different way to the primary key - but that doesnt mean 
that it  has to be done this way.  Anything that works for the 
Primary key will also work for the Secondary key - and - vice 
versa!  I'm just using the Secondary key to introduce another 
capability.

[name.search]
    print #w.star, "!contents?"
    input #w.star, mainActor$

    key2$ = trim$(mainActor$) + chr$(0)
    CallDll #db, "GetWild", hx2 as word, key2$ as ptr, cNum as long
[test.name]
    if cNum < 2 then
        Notice "No (more) Movies Actors match this search"
        goto [loop]
        end if

    get #f, cNum
    gosub [display.movie]
    goto [loop]

[next.name]
    CallDll #db, "GetNextWild", hx2 as word, cNum as long
    goto [test.name]

We'll ease into this gently.  If you look at the code for 
[name.search] it is very similar to the code for the [next]
button of our main index.  But instead of using GetFirst
and GetNext - we are using GetWild and GetNextWild.  

In its simplest form, it works the same way as the GetFirst
and the GetNext.  That is - you enter the first part of an
actor's name - and it will get the first actor that 'matches'
the part of the name you supplied.  When you click on 'Next 
Name' it returns the next record that matches.  You will 
notice that these records are presented to you in a different 
sequence to the way you see them with the Main Index.  That's 
because they are sorted on Actor name - NOT - Cassette number.

Still with me????  I hope so - because we are about to get very
fancy.  What I'm about to show isn't exclusive to Brosco's
DBdll - but you wont find it in many - this is VERY special
stuff!!!

Your Search key for GetWild can include WildCards - just like 
a DOS Directory search.  There are 3 special characters that
you can use:   *, ? and !

The first two ( * and ? ) you are probably familiar with.

If I enter a search key of:
Peter*
I will be returned all the actors that have 'Peter' in the
first part of the key - regardless of what follows that part 
of the name.   If I enter a search key of:
*Sellers
I will be returned all the records that have 'Sellers' at the 
end of the key - regardless of whats in the first part of the
key.

If I enter a search key of:
*Newton*
I will be returned all the records with 'Newton' anywhere 
within the key.

A search key of:
*New??n*
I will get records that have 'New' somewhere in the key - but
will also have an 'n' exactly two positions after the 'New'.

And finally - here's the real biggy!  A search key of:

!*peter*

will return all the records that have 'peter' in the name,
but it will not matter whether its in Uppercase, lowercase or 
a mixture of the two.  The '!' in the first position of the
search - says ignore Upper/Lower case for this search.


The full source for this program is contained in the attached
NL13.ZIP.


Now - have you understood all of this?  If not, please use my 
message board at the above site to post your questions.

If you have got it - why not try to add another secondary index
so that you can perform browsing on the Movie titles as well.  
If you can manage to do that - you should feel somewhat confident 
to branch out and build the best Database program that the LB
community has ever seen!
---------------------------------------------------------------

3) Data Dictionary - DB Application Generator.

OK, this and previous newsletters, showed how easy it is to
index your Random files using DBdll.  Now, while each of the
functions on there own are easy to understand - but there is a
lot to remember when putting together a complete application.

I have included a program in NL13.ZIP called DD4.BAS.  This
program allows you to enter a File name, the FIELD names and
lengths and indicate the KEY fields.  It will then Generate a
fully functioning LB program to Add, Update, Delete and Search
records on your database.

This program should not be considered as the final application.
It should be used as a template for you to build your application.
There are many things that you can do to improve on the generated
program.

a) Use more meaningful descriptions in the StaticText controls.
b) Add some data 'validity checking' to ensure that only 'valid' 
   data gets stored on the data base.
c) Maybe use Ctl3d to improve the window appearance.
d) Improve the screen layout of the data fields. 

etc. etc.

I hope you will find this program useful.


---------------------------------------------------------------

I would expect this to be the last issue on the series on RANDOM 
files for a while.  Dean and I will be working on a new and
improved dbdll that will be even more powerful.  I will give you 
some more details when we get further advanced with the design 

Its now time to start thinking about some new topics - ideas and
suggestions from you are most welcome.

Next issue I had intended to do an article on Status bars and 
progress bars - but I received a contribution from David Szafranski.
And all I can say is: "WOW!!!!!!".  You can download this 
excellent demonstration from my 'Brosco's Seal of Approval Page' at:

http://users.orac.net.au/~brosco

And while you are there - take a look at the contribution from 
Destiny.  Here is an excellent example of using 'Tabbed Data Entry'
in a Dialog window.  I got Destiny to allow me to publish this 
program before it was completed.  At this stage the program only
shows you how to handle the 'Tabbed Display'.  Destiny is currently
working on incorporating DBdll and Random files to store the data.  
I look forward to seeing the final program.

I'm taking a small holiday from the Newsletter for a week or two.
But don't be concerned - Alyce has promised to produce the next 
issue for you.  And since Alyce doesn't need the services of Brosco
to turn her writing into comprehensible English - I have no idea 
what the topic will be.


__________________________________________________________________


PLEASE NOTE: DBdll is COPYRIGHT and will eventually be 
distributed as SHAREWARE.  I have made it available FREE to
the Liberty Basic community for your own personal use.

Please observe the following conditions for your use of LBdll.

1) You must NOT distribute it to ANY other parties in any form.
2) Any commercial product produced using DBdll requires a
   Registered version of the product.  Contact me for details.
3) Failure to comply with these conditions will incur the wrath 
   of 1000 Gods - plus my personal curse of:
         "May your toenails forever grow inward!!!!!!!"
   and to the older Ozzie users, who may remember the 'Aunty Jack'
   show: "I'll rip your bloody arms off!"



---------------------------------------------------------
 Newsletter written by: Brosco.
 Comments, requests or corrections mailto:brosco@orac.net.au

 Translated from Australian to English by an American:
 Alyce Watson -  Chief Editor.  Thanks Alyce.
---------------------------------------------------------
