187 replies to this topic

[Kilburn]

As the topic I created is not very active (not at all!! ), I prefer posting here...
I am working on statistic regressions (linear, quadratic, etc...). I managed to add linear regressions, but I don't know the algorithm of some of these regressions.

I noticed that you often use:

function myfunction(a,b,...)
 local c=0
 if arg["n"]==1 then
  c=arg[1]
 end
end

I use that:

function myfunction(a,b,c)
 if not(c) then
  c=0
 end
end

It is faster, no?

[PAP]

It is faster, no?

It is somewhat simpler, but not faster. Quite often, I need to modify an optional argument, and, to save computation time, I want to modify the optional argument only if the user has actually used it. For example, I may need to modify "c" to "{c,c}", if the user has passed a number for this, while a table of two numbers is expected. Of course, this can be done in your code, if you add an "else" branch, but then you have to modify a dummy argument (Lua may not care if you do so, but I do care - I think that it is not a good programming habit). On the other hand, using nested ifs that access the "arg" table looks more preferable and clearer to me, because there is no need to modfy a dummy argument - but this is a matter of personal taste...

[omegavirus]

PAP wont answer because he is in Paris for a week or something like that, check the version of your CPLua =)

[Kilburn]

why i can't run LuaNumAn 1.30

every program can't run adn show screen in picture



i can run 1 program is Demoplot.

Why?
sorry i'm newbie for CPLua

These are not really programs. In fact, these programs are packages, like "draw" or "table". For example, you must use require("LuaNumAn/Krone") to include all the functions contained in the file "LuaNumAn/Krone".

In the "LuaNumAn" folder all the files which start with a "D" (for Demo) are launchable programs.

PS:In function require() you must use '/' instead of '\'.

[Orwell]

These are not really programs. In fact, these programs are packages, like "draw" or "table". For example, you must use require("LuaNumAn/Krone") to include all the functions contained in the file "LuaNumAn/Krone".
In the "LuaNumAn" folder all the files which start with a "D" (for Demo) are launchable programs.

That's right, these libraries have to be called by a script, you cannot run them directly
Try the demo programs to see how to use them.

This error happens because those files try to export some functions to the calling script, and there isn't any; I will try to make CPLua ignore the export command in those cases.

PS:In function require() you must use '/' instead of '\'.

Both slashes should be recognized

Btw Kilburn: I know that the files you load with require() are executed, but they should not contain any instructions; it is only made to export some variables or functions to the calling chunk. Try to write all "require" operations at the beginning of each script, and not between instructions because you cannot be sure that the file will be executed each time you call require. There is a doscript() function to execute instructions in another file, and it is more suitable

[PAP]

Hi all, PAP is baaack!
Happy to be here again, but unhappy as well, since I cannot enjoy the french "brasseries" anymore ("un demi s'il vous plait"). The french aperitif named "Kir" was excellent, and the "pave au poivre" was delicious.
As with my previous visits in Paris, I came back with a suitcase full of books about Numerical Analysis (there is a bookstore in "Boulevard de Saint Michel" that has everything you may ever need in Mathematics).

This error happens because those files try to export some functions to the calling script, and there isn't any; I will try to make CPLua ignore the export command in those cases.

I don't think that it is a good idea. This error occurs when the user attempts to run a CPLua chunk containing "export" functions, as mrKaqz did. The error warns the user that the chunk is actually a "library" made to be used by runnable programs, and it should not be run dietly. I think that it will be more confusing to the new user, if CPLua ignores the export functions if the chunk is executed directly.

Try to write all "require" operations at the beginning of each script, and not between instructions because you cannot be sure that the file will be executed each time you call require. There is a doscript() function to execute instructions in another file, and it is more suitable

Writing all "require" operations at the beggining of the chunk is also convenient to inspect the dependencies of the current chunk. I do it systematically in all my library chunks. In my opinion, such a chunk should only contain function (and constants) definitions. Anything else should not be issued there.

[Orwell]

Welcome back You seem to have enjoyed your trip

Well, about this export "error": I agree that we should not be able to run a library, but following this logic a runnable script should not export anything (since any shared element between different chunks should be put in a shared library; this is why I wanted to distinguish the "scripts" and the "functions" in the first versions of CPLua)... But if we do that, there will be a lot of possibilities that won't be usable anymore

We could imagine a stand-alone script that would also export some useful things for the eventual other scripts that run it. In that case the export operation should be ignored when this script has no parent

Or think about a script that would return a special function, which depends on some given arguments: you cannot give arguments when you use the "require" function, thus you have to use "doscript" and run the corresponding script, which will export the function. In some cases it is also convenient to replace a function by another at runtime, and since "require" should only be used at the beginning of each chunk, the "doscript" function would also be used.

We must find a way to preserve the robustness of the code and the different possibilities of the Lua language

[PAP]

Or think about a script that would return a special function, which depends on some given arguments: you cannot give arguments when you use the "require" function, thus you have to use "doscript" and run the corresponding script, which will export the function.

See how the "Krone" library is implemented, it's the way I have found to overpass this problem: there is a callable function that gets as arguments everything it is needed. All these arguments are copied to variables, which are local to the library chunk. Then the special function (also part of the chunk) uses them to do its job. I needed to implement it this way because the special function should only have one argument. The rest of the arguments actually needed are passed that way, without using the doscript function. The robustness is preserved, but you need to copy some arguments to local variables, and this may be memory consuming sometimes.

In some cases it is also convenient to replace a function by another at runtime, and since "require" should only be used at the beginning of each chunk, the "doscript" function would also be used.
We must find a way to preserve the robustness of the code and the different possibilities of the Lua language

In general, I agree, but I'm afraid that requiring such a flexibility means that we will also increase complexity. For example, Fortran 95 is very flexible on defining modules, but it is also a complex language, and many of its features are scaring novice programmers; <{GNULINUX}> offers extreme flexbility, but it is also a rather complex operating system, so that a novice user may find it inaccaptably difficult. But I'm not sure, maybe there is a way to increase flexibility without horribly increasing complexity.

[unique33]

maybe this Resources can help some one who is interested in numerical methods :

GAMS :
the guide to available mathematical softwares currently contains information on some 9000 software modules from about 80 packages found at NIST (National Institute for standard and technology)
and on netlib
http://gams.nist.gov/


NAG :
(Numerical Algoritms Group )
large library of Fortran 77 subprograms
Fortran 90
C
,...

http://www.nag.com

[PAP]

GAMS :

This is an extremely large sotware collection, but C/C++ programers be aware: The majority of the modules you can find there are written, of course, in Fortran .

NAG :(Numerical Algoritms Group )

Hmmm, this is commercial software, and only a trial version is available for download, unless you pay for it. However, you can find good alternatives in GAMS. For example, the free package "QuadPack" is extremely powerful for integration, and the free large package "LAPACK" is a perfect solution for computer Matrix Algebra.

[Orwell]

Btw PAP, by taking a deep look on your LUdecom code I have some remarks/questions for you:

• You are using strict equality to check if an element is equal to zero; but shouldn't you rather consider a small tolerance around zero (like -1e-14 -> 1e-14), because of the possible round errors of the 'double' type?
• LUdecompose prints an error if the matrix is singular. But what if we want to compute the determinant of any matrix (even singular), for example just to check if the matrix is singular or not?
• The use of a 'Bp' table in your LUsubstitute functon is rather memory inefficient. I think that using B[indx[i]] instead of Bp[i] would be quite better
• in the "Sub-diagonal division" part of the LUdecompose function, the denominator is the same for each iteration, so you could maybe place it in a temporary variable instead of accessing the table each time (it should be a little bit faster)... But this is rather a optimization detail

[PAP]

Btw PAP, by taking a deep look on your LUdecom code I have some remarks/questions for you:

Wow, it's the first time that someone makes such a "deep" test in a LuaNumAn library. Thanks!

You are using strict equality to check if an element is equal to zero; but shouldn't you rather consider a small tolerance around zero (like -1e-14 -> 1e-14), because of the possible round errors of the 'double' type?

I know, I know, but, unfortunately, a "small tolerance" is not a solution in this problem, since the matrix may be close to singular, but not exactly singular. For that reason, most Numerical Analysis books suggest to compare strict equality in this case. Of course, you are right, this means that roundoff errors may lead to treat a singular matrix as a matrix "close to singular", but it is well known that there is nothing "magical" in Numerical Analysis. Note that even in the book Numerical Recipes, which is considered by many people as "the bible", LU decomposition is implemented by comparing strict equality; the french book I mentioned in a previous post also compares strict equality. Nevertheless, and despite on what the books say, I'm really insure which implementation is better for this .

LUdecompose prints an error if the matrix is singular. But what if we want to compute the determinant of any matrix (even singular), for example just to check if the matrix is singular or not?

Well, if the matrix is singular, it cannot be inverted, the corresponding system of linear equations does not have exactly one solution, and the determinant is equal to zero.

The use of a 'Bp' table in your LUsubstitute functon is rather memory inefficient. I think that using B[indx[i]] instead of Bp[i] would be quite better

I'm afraid that in this case the code will be slightly slower, since indx[i] should be evaluated again and again, within a loop that will need to evaluate B[indx[i]] instead of Bp[i]. For that reason, I created a local table Bp (B permutted). Note that this table is actually the right-hand side of the system of linear equations, and its use is not so memory consuming; how many equations may have a system that you will solve in ClassPad? 10? 100? 1000?

in the "Sub-diagonal division" part of the LUdecompose function, the denominator is the same for each iteration, so you could maybe place it in a temporary variable instead of accessing the table each time (it should be a little bit faster)... But this is rather a optimization detail

Hmm, I'm really unsure if this will make the code faster. I thought that memory access for a table element isn't noticeably slower than memory access for a scalar variable, considering that temp=L[j][j] will take some time to be evaluated too. But you are right, this is an optimaization detail (but details are useful too). Unfortunately, we can't measure execution time to see if it makes some difference in large linear systems.

[Orwell]

Well, if the matrix is singular, it cannot be inverted, the corresponding system of linear equations does not have exactly one solution, and the determinant is equal to zero.

I know that, but suppose that someone writes a program allowing the user to enter a matrix, and computing the determinant. In general, the fact that the matrix is singular is not an "error", it is a particular case... If your function 'LUdeterminant' is used in the program there will be an error message, and it is not allways wanted by the programmer

I'm afraid that in this case the code will be slightly slower, since indx[i] should be evaluated again and again, within a loop that will need to evaluate B[indx[i]] instead of Bp[i]. For that reason, I created a local table Bp (B permutted).

Well I understand, but in your LUsubstitute function each element of Bp is called exactly once

Hmm, I'm really unsure if this will make the code faster. I thought that memory access for a table element isn't noticeably slower than memory access for a scalar variable, considering that temp=L[j][j] will take some time to be evaluated too. But you are right, this is an optimaization detail (but details are useful too). Unfortunately, we can't measure execution time to see if it makes some difference in large linear systems.

You could write a small script to call it a thousand times, and measure the execution time by hand
But I think that it will certainly be faster, since there is only one operation needed to read the value of 'temp', however to access 'L[j][j]' the interpreter has to:
- read the value of 'L' and identify it as a table
- access the element 'j' of this table and identify it as another table
- access the element 'j' of this other table

[PAP]

I know that, but suppose that someone writes a program allowing the user to enter a matrix, and computing the determinant. In general, the fact that the matrix is singular is not an "error", it is a particular case... If your function 'LUdeterminant' is used in the program there will be an error message, and it is not allways wanted by the programmer

Good point. Initially, I designed LUdecompose as a function solely for solving linear systems, where a singular matrix is certainly an error. Then I decided to use it for computing the determinant and the inverse, but the error message was untouched. I'll change it, so that LUdecompose will return an empty LU matrix, if the matrix A is singular. Then LUdeterminant will return 0, if its input argument "LU" is {}, and LUsubstitute and LUinverse will return an error message in this case.

Well I understand, but in your LUsubstitute function each element of Bp is called exactly once

Ooops, you are right. I'll change it immediately. As it is now, it is indeed a (small) waste of memory and time.

You could write a small script to call it a thousand times, and measure the execution time by hand

I deed such things in previous programs. It's the only way to have some estimation of execution time, but it's annoying.

But I think that it will certainly be faster, since there is only one operation needed to read the value of 'temp', however to access 'L[j][j]' the interpreter has to:
- read the value of 'L' and identify it as a table
- access the element 'j' of this table and identify it as another table
- access the element 'j' of this other table

Ok, I'll change it as well. All changes will be present in the next version, were everything will be done by matrices, not tables.

[Orwell]

I deed such things in previous programs. It's the only way to have some estimation of execution time, but it's annoying.

I was thinking about some useful timing functions (something like tic() and toc()), but unfortunately there is currently no way to measure time with good accuracy (we can just measure seconds but not milliseconds etc)...

[PAP]

I was thinking about some useful timing functions (something like tic() and toc()), but unfortunately there is currently no way to measure time with good accuracy (we can just measure seconds but not milliseconds etc)...

If the time can be measured accurately, these functions will be better than nothing, even if the time returned is accurate to seconds (and not milliseconds).

[Kilburn]

Can't you use SH3Timers?

[Orwell]

Those timers are still in an "experimental" step: I will wait a little bit before using them definitely in CPLua

[PAP]

Version 1.40 is out.

Although I was initially intended to release a new version after changing everything in accordance with the new features of CPLua 0.73, this seems to be a rather complicated task, and it will take some time, since it will need extensive checks. Furthermore, I wanted a "final" release, before performing such "deep" modifications. So, I finally decided to release the current version, which is still written for CPLua version 0.72 (but, of course, totally compatible with version 0.73 as well). This is the last version where Lua tables are used; the next version will extensively use matrices instead.

Major changes:
(1) A method for numerical integration via the adaptive trapezoidal method has been added (TrapAdapt).
(2) LU decomposition has been slightly upgraded, and it now handles singular matrices more conveniently (many thanks to Orwell for his useful remarks on this subject).
(3) The function Romberg has been slightly changed. It now returns the estimated integral, even if the desired accuracy has not been reached (in this case, a warning message is also printed).
(4) A menu-driven demo script (DemoAll) has been added; it demonstrates all capabilities of the current version, and may be useful for the begineer in LuaNumAn.

If you are interested, LuaNumAn version 0.40 can be downloaded here. As usual, the documentation has been modified, and explains all new features and changes.

Note: The version numbering scheme I finally adopted is "a.bc". A change in "c" means that only minor modifications have been made; a change in "b" means that at least one new numerical method has been added; a change in "a" means heavy modifications, or a "milestone" in LuaNumAn developement.

[rafapa]

Dear PAP,
I believe I have found and small typo in the LuaNumAn documentation (1.40 version). On page 15 of 25, when you describe the LUsubstitute function, you say "REMARKS: The function LUdecompose must be called before calling LUdecompose." The last LUdecompose should be LUsubstitute, isn't it?

I also have a couple of questions regarding LMfit. When you talk about sigma, this is the same to what other people call wheigths? (ok, maybe some square root is involved). The other one is: it could be possible to add the calculation of the correlation matrix for the fitted parameters?

Thank you for the attention and for the developement of LuaNumAn.

[PAP]

Thanks for your remarks. They are all useful to me.

I believe I have found and small typo in the LuaNumAn documentation (1.40 version). On page 15 of 25, when you describe the LUsubstitute function, you say "REMARKS: The function LUdecompose must be called before calling LUdecompose." The last LUdecompose should be LUsubstitute, isn't it?

Indeed, you are absolutely right. This is a "copy-paste-and-forget-to-modify" typo; I have corrected it. Let me know if you find other typos as well. Although I have spent a lot of time for correcting and improving documentation, it seems that making typos is inevitable, no matter the effort.

When you talk about sigma, this is the same to what other people call wheigths? (ok, maybe some square root is involved).

sigma is the standard deviation, and it is related to the "weight" as sigma=sqrt(1/weight), where weight>=0. "Unweighted" measures have sigma=1, which is the default in LMfit. I can understand your question: in some books, "standard deviation" is reffered to as "weight", and this is very confusing. I will add a remark in the documentation explaining this.

it could be possible to add the calculation of the correlation matrix for the fitted parameters?

In fact, the correlation matrix is computed in LMfit, but it is not returned to save memory, since usually it is not needed. I can modify LMfit to return the correlation matrix, if you think it is useful. However, it's easy to do it yourself: In the LMfit code, the correlation matrix is the matrix C, calculated in the function Confidence, and it is equal to the inverse of the matrix G, defined in the function LMfit.

Thank you for the attention and for the developement of LuaNumAn.

You are welcome. Nice to know that you are interested on this project. Again, thanks for your useful remarks.

[PAP]

LuaNumAn 1.50 is in heavy developement now. I'm not going to announce all the new features (I'll do it in time). I just want to say that LuaPlot now supports "infinite" plots, similar to those proposed by unique33. They are very useful to inspect the "behavior" of a function as x goes to +infinity or -infinity. Consider the following screenshots:



First screenshot: "infinity" plot for the functions exp(x), ln(x), and 1/(x*(x^2-1)).
Second screenshot: "infinity" plot for the functions x^0.25, x^0.5, x, x^2, x^5, and exp(x).
Grid lines denote x=-1,x=1,y=-1,y=1. You can see that these plots are heavily "streched" for large |x|, since the interval [0,1] for x (or y) is "equal" to the interval 1+infinity). This is due to the transformation made in x- and y-axis. As unique33 already pointed out, these plots cannot show function details for large x-values. Nevertheless, they are useful. For example, in the first screenshot you can clearly see that exp(x) tends to infinity faster than ln(x) - a well-known fact. You can also see the limits of the function 1/(x*(x^2-1)), as x goes to the singular points x=-1, x=0, x=1. In the second screenshot, you can see how n affects the way that x^n goes to infinity, as x goes to infinity.

Despite the above satisfactory results, I'm not sure if the transformation I have used is identical to that used in Maple. So, unique33, can you do me a favor? Can you reproduce (and post) these graphs, as obtained by Maple? This will greatly help me to figure out if the transformation used in LuaPlot is identical to the Maple's transformation. Please use the same functions as above. Furthermore, if you have more information about Maple's infinity plots, please let me know.

I'm now working on a more flexible transformation, able to show more function details for large values of |x|...

[Kilburn]

I just want to say that LuaPlot now supports "infinite" plots, similar to those proposed by unique33. They are very useful to inspect the "behavior" of a function as x goes to +infinity or -infinity.

It implemented this in my Graph library a long time before you... but I think I will stop the project. I have more important things to do (learning my lessons, for example! )
And next holydays, I will try to develop a Graph35 emulator.

[PAP]

It implemented this in my Graph library a long time before you...

"long time before you"? What's this? A comparison? A competition? Allright then: Implementing robust infinite plots is more complex than you think. No offense, but you probably think it's easy because you are too young to know much about Mathematical Analysis, especially transformations. You are obviously a talented programmer, but this is not enough: you need to learn more Mathematics , and you will probably do so... in time.
No offense, again, but if you really want a competition, try to develop something like LuaNumAn .

[unique33]

First screenshot: "infinity" plot for the functions exp(x), ln(x), and 1/(x*(x^2-1)).
Second screenshot: "infinity" plot for the functions x^0.25, x^0.5, x, x^2, x^5, and exp(x).
Grid lines denote x=-1,x=1,y=-1,y=1. You can see that these plots are heavily "streched" for large |x|, since the interval [0,1] for x (or y) is "equal" to the interval 1+infinity). This is due to the transformation made in x- and y-axis. As unique33 already pointed out, these plots cannot show function details for large x-values. Nevertheless, they are useful. For example, in the first screenshot you can clearly see that exp(x) tends to infinity faster than ln(x) - a well-known fact. You can also see the limits of the function 1/(x*(x^2-1)), as x goes to the singular points x=-1, x=0, x=1. In the second screenshot, you can see how n affects the way that x^n goes to infinity, as x goes to infinity.

Despite the above satisfactory results, I'm not sure if the transformation I have used is identical to that used in Maple. So, unique33, can you do me a favor? Can you reproduce (and post) these graphs, as obtained by Maple? This will greatly help me to figure out if the transformation used in LuaPlot is identical to the Maple's transformation. Please use the same functions as above. Furthermore, if you have more information about Maple's infinity plots, please let me know.

I'm now working on a more flexible transformation, able to show more function details for large values of |x|...

maybe next monday I will be at home ,I will post them then. I,m so busy these days , I have plenty of Midterms!
also consider this function ( If I remember truely)
8*sin(x)-x
If you plot it verticaly, you will see that you can see more details !
It,s because the classpad LCD x range is lower than the y .

[Kilburn]

You are obviously a talented programmer, but this is not enough: you need to learn more Mathematics , and you will probably do so... in time.
No offense, again, but if you really want a competition, try to develop something like LuaNumAn .

No, you are right, and I am just 14, so I understood nothing in your source code.
...and thanks for the "talented programmer"

[TacoFred]

Just to ask, what are your opinions on PC CASs (Maple, Mathematica, etc.), which would you prefer to use?

[PAP]

Just to ask, what are your opinions on PC CASs (Maple, Mathematica, etc.), which would you prefer to use?

Mathematica and Maple are the most known PC CASs; however, there are plenty of them, such as MuPAD (also commercial). Among the commercial applications, Mathematica seems to be more powerful and advanced, according to some tests I have made. However, I haven't used Maple for a long time, so I'm not quite sure.

There are also several free PC CAS alternatives, such as Axiom, Yacas, and Maxima.
Axiom is based on older commercial software; it is very powerful, but, for the moment, it has a rather poor interface.
Yacas is a command-line CAS; its main advantage is that it can even be installed on a palmtop.
Maxima is an old CAS, now released as OpenSource. It's rather poor, but it has powerful support for tensor algebra.

Obviously, there is no perfect CAS.

[unique33]

here is the maple screen shots :

[unique33]

Despite the above satisfactory results, I'm not sure if the transformation I have used is identical to that used in Maple.
I'm now working on a more flexible transformation, able to show more function details for large values of |x|...

yes , you are right , they are not identical :

[PAP]

yes , you are right , they are not identical :

Thanks for the graphs. As I suspected, the transformation used in Maple is not identical to the one I have adopted in LuaPlot: Maple's graphs are less "streched" for large values of |x|. Your graphs will help me figure out what's happenning. However, if you have any additional information, e.g., from maple's manual, let me know.

[unique33]

Thanks for the graphs.

you,re welcome.

As I suspected, the transformation used in Maple is not identical to the one I have adopted in LuaPlot: Maple's graphs are less "streched" for large values of |x|. Your graphs will help me figure out what's happenning. However, if you have any additional information, e.g., from maple's manual, let me know.

there is no more information in maple manual . I posted all the information which was in maple help before. ( just it say it is useing arctan , nothing more )

[PAP]

there is no more information in maple manual . I posted all the information which was in maple help before. ( just it say it is useing arctan , nothing more )

I show your previous post, but as we saw from the examples above, Maple does not use a simple arctan transformation. Although it is not necessary to copy the Maple's way to plot "infinity plots", I will try to figure out the exact transformation used to Maple.
Btw, did you noticed that LuaPlot doesn't plot vertical lines at singular points (while Maple does)? It's one of the new features added .

[unique33]

Improved Infinity plot:

If f(x) = x ,ln(x) ,exp(x)
then
arctan(1/8*f(x)|x=8*tan(x))

else
arctan(f(x)|x=8*tan(x))
end

[unique33]

here is some screenshots :

[PAP]

here is some screenshots :

Well, from the screenshots it is obvious that I was right: an infinity plot 'streches" the curve severely for large values of |x|, where "large" |x| means above 3~4. In other words, if the function changes rapidly for large values of |x|, the graph is totally messed up. Nevertheless, infinity plots are useful for smoother curves. Btw, the "improved infinity plot" you proposed only fixes specific functions (x, ln(x), exp(x)). It gave me, however, an idea on how to proceed. Unfortunately, I'm quite busy now. Don't expect LuaNumAn version 1.50 before Christmas . It has many new things added, and they need a lot of time to be tested...

[-Tom-]

Pap, could You recommend me please good site with curve fitting? You used L-M method in LuaNumAn, but I would like to check other...
Thanks in advance!

[PAP]

Pap, could You recommend me please good site with curve fitting? You used L-M method in LuaNumAn, but I would like to check other...
Thanks in advance!

The Levenberg-Marquardt (LM) is the de facto method for non-linear fitting (but it is also the most difficult method to implement). However, if you just want to do somekind of linear fitting (such as linear, logarithmic, or exponential fitting), you can also use simpler methods (but much less powerful than LM). I have included LM in LuaNumAn as a robust method for any kind of fitting, and I'm not planning to add simpler methods until version 2.xx, since LM does the job efficiently. Note that data fitting is often confused with data interpolation, but they're different things: in short, data fitting is used when you know a theoretical curve that fits your data, while interpolation is used when you don't know such a theoretical curve. For this purpose, I have already implemented cubic spline interpolation, which will be available in version 1.50.
Now, about the web sites you asked, my opinion is this: do not trust web sites dedicated to Numerical Analysis lessons; I'm sure that there are good sites, but I have only seen poorly documented sites, which cannot be compared with a good book. So, I'm posting the titles of some excellent books, where data fitting is covered:
(1) Brian Bradie, A friendly introduction to Numerical Analysis, Pearson Prentice Hall, 2006. Despite its title, this book is more than an "introduction". It also has excellent examples, taken from physics, chemistry, biology, economics, etc. A "must have", although the LM method is missing...
(2) J-P. Nougier, Methodes de calcul numerique Vol. 2, Hermes Science, 2001. This is the book. Its mathematical level is a little bit "heavier" than the others, but it covers data fitting (as well as other topics) very extensively. I hope you speak french; if you don't, you can't read my favorite book .
(3) J. H. Mathews & J. D. Fink, Numerical Methods Using Matlab, Pearson Prentice Hall, 2004. Very useful book, even if you don't use Matlab. Simple data fitting is covered extensively, with many examples, but the LM method is completely missing.
(4) W. Press, S. Teukolsky, W. Vellering, B. Flannary, Numerical Recipes (2nd edition), Cambridge University Press, 1992. This is the well-known best-seller, but, in my opinion, it is not as good as many people think. In fact, it is widely used simply because ready-to-use programs are included (they can also be purchased separately). It covers plenty of topics, but it does not explain how each method works (well, it does, but only in short). If you are interested, you can also read the book in www.nr.com.

[-Tom-]

Thanks again!
Like always topic fully covered! Nothing left, just reading!