FM Bench Detective
brought to you by HOnza Koudelka from 24U Software
In the first two videos you could learn about my experiences with optimizing FileMaker solutions. I have shared my discoveries and advices what to do and what to avoid if you want to speed up your solution efficiently. If you have not seen the previous videos, I suggest you check them out before watching this one. Just click the links in the top of this page to get directly to the other videos.
In this video I am going to walk you through 24U FM Bench Detective…
This is the actual tool that my team uses to benchmark our own information system in 24U. We benchmark about 1,000 scripts and generate over 300,000 measurement records every day. Thanks to that we are able to constantly watch and improve performance of our system.
But I am not just going to make you spend 12 minutes watching another video you’ll quickly forget…
I am actually going to give you a copy of 24U FM Bench Detective for free!
Just register below and get instant free access to both the video and your own copy of FM Bench Detective.
{ 24 comments… read them below or add one }
HOnza,
We’re really excited to be using FM Bench Detective and working with you and 24U Software. We’ll keep you posted on our experience using FM Bench Detective as part of our development toolset.
Best regards,
Allen
Hey Honza
The explanation video stops after 12 seconds?
Cheers
Steve
Seems like sometimes the video stops downloading. Maybe too many people watching it? That woukd great
Or maybe JWPlayer just does not prefetch enough of it before starting the playback. Anyway, reloading the page usually helps…
Whoops – works fine now after I DL’d the file.
I have to say, I am feeling VERY excited about doing optimisation on my solution.
Cheers
Steve
HOnza (and team),
What a fantastic tool! I can’t wait to see the final product. Well done!
Aaron
Thank you, I am glad you like it. Have you already tried to use FM Bench Detective to do some optimization? I am looking forward to hear about your own experience.
For your inspiration, I have used FM Bench Detective to optimize one custom function last week. And I have made it evaluate up to hundreds times faster. I have explained the optimization in this blog article: http://honza.24usoftware.com/custom-function-optimized-to-evaluate-hundreds-times-faster
I keep track of my Custom Functions in a database. When a Custom Function is revised, I compare some “Sample In” results to previous “Sample out” results to see if I have caused an error during the revision. I also want to compare the execute times to the previous revision. That lets me know if I have optimized or slowed down the Custom Function. I used to use “Extra Suites” on the Mac for this but it was very cumbersome as I had to find and remove my own overhead. I could only get an approximate overhead value. FM Bench will be ideal to re-implement this in a clean manner.
I would suggest that developers may want to augment FM Bench with histogram graphs via the xmCHART plugin to get sort of a visual “signature” of what the script “looks like” from an execute-time perspective.
Also, to show graphs that normalize the execute time to the number of records in the Found Set.
My personal experience has shown me that it is faster to use Custom Functions to examine the Found Set and return pertinent “indexes” (record numbers) that I want to operate on. For instance, to remove duplicates, it “seems” faster to (1) sort records and (2) call a Custom Function that returns indexes of the duplicates based upon multiple field matches and then (3) use a Script Loop — starting at the last record — to “go to” each record of each “index” and delete them. For example, if there are 10,000 records to delete duplicates and there are 100 duplicates, the Script Loop only loops through 100 records to be deleted instead of 10,000 records. My tentative conclusion is that Looping through records in a script is slower than using “GetNthRecord” in a Custom Function. Now, I will be able to know if this as fact or not.
Have you ever wanted as script to gather a list of all values of “Field 1″ only if “Field 2″ is “true”? One way is to define another relationship with two keys and this is easy and fast. However, if this script is only used occasionally, and your relationship graphs is very crowded, you may want to use a Custom Function that gathers data for “field 1″ conditionally only if “Field 2″ is the target value. This eliminates the need for a relationship. Using FM Bench, I will be able to quantify this method to see how much slower it is than using another table occurrence.
I’m sure I’ll be back with some more comments…
Thanks for this plugin HOnza!
Steven Sanders
Thanks for such a detailed comment and suggestions. I am glad to see some of your “feature requests” matching the feature set of the whole FM Bench product we’re about to release soon
I am looking forward to your future comments…
HOnza,
thanks for sharing your ideas and profiling-tools!
It’s really great inspiration on optimizing FileMaker-Solutions.
regards, Marcel
Hey optimizers, thanks for all your feedback and questions you’re sending me by e-mail. Please don’t hesitate to share your opinions here in the Comments section…
I have just uploaded FM Bench Detective 1.0.1. Click the download button above to download the update.
In the new version:
• Improved handling of timestamps with non-US date/time formats (replace your FMBench_Log with the new version)
• FMBench_LogCustomEvent can now be used to benchmark recursive custom functions
• Added sample recursive custom function RepeatChar to demonstrate FMBench_LogCustomEvent
• FMBench_LogCustomEvent fixed to not interfere with FMBench_LogScriptEvent
• Improved handling of special characters in FMBench_LogScriptEvent and FMBench_LogCustomEvent parameters
Intriguing idea!
Hi Honza,
the video stops after 20-25% …
best regards – Dick
Please try to reload the page or rewind the video a little bit. For some reason the player stops playing the video wen the pre-fetching mechanism fails to load the video data fast enough for continuous playback. but as far as I know it worked fine for everyone after reloading the page.
I finally got time to check out your solution, and I noticed the FM Bench Analyzer is not there. I assume that is intentional since I don’t see any other comments about it.
It’s funny – I actually did this for myself about six months back when I had performance issues, only without the use of custom functions or plugins. I just send the name of the script as a parameter to the logging script, then it logs it in my corresponding file, similar to yours. Only I was storing a start/end time both on the same record – so I’d find the record I created at the beginning of a script, and add the end time when it ended.
Anyway, I never developed a good interface for me to analyze my own data – I just perused through the data to get a feel for where my bottlenecks were. So I was intrigued by your solution since I saw the Analyzer in the video. Plus, you grab more info than I do, which could be useful.
So, what’s the word on the Analyzer?
I have a huge solution for one of my clients and I’m trying to make a cost reporting tool in the solution more efficient. Also, it’s server-side, so in looking at your plugin’s external functions, I see I need a serial number to register. I assume this is obtained by purchasing the plug-in, correct?
Great job on this and thanks for sharing with everybody. Thanks for your help.
Where does the plugin get its time data from as it is an hour out on my machine in the UK??
Toolbox_GetTimestamp should be giving the same result as Get(CurrentTimeStamp), just more precise. Do you get different results?
Calculations, autoenter values and look-ups? Will they be measured in any way? I understand you measure every step in a script, and that is very good. Sometimes, cascading calculations can be the most timeconsuming part of daily use in a database. Any plans to measure them?
Thanx for all your effort HOnza! /Arild
You can use the FMBench_LogCustomEvent function to measure these things. Just be careful as you can easily slow down your solution by the overhead of the measuring function itself.
I do advise to keep measuring scripts forever, but for calculations I recommend the opposite. Remove the measurement as soon as you’re done with your tests.
Hey optimizers,
FM Bench Detective has just been updated to version 1.0.2.
Click the download button above to download the updated version.
Enjoy the new version and keep your eyes open, the release of the whole FM Bench is finally coming in a few days.
Hi Honza, thank you for your pointers and toolkits, I’ve succeeded in adding the log script steps into my development project, and thrown together a rough analysis of some data, very interesting results! and as you suggest, retaining this in the scripts of the solution as it goes live, would allow changes due to record usage to be monitored and reviewed as required. Very handy!
I have been using an internal log table in a couple of my solutions to watch for which users sign in and when as well as some timing for some of the report scripts they run, but had not played with your toolbox script which allows a better timing feature. Looking forward to giving this new tool a try and congratulations on not only releasing it to developers but allowing a free option to encourage us to try it out and all benefit from better measuring our solutions.
Hi, the free copy is a very good idea, but where is there FM Bench Analyser? This soulotion is useless without this Database!!!
Thanks for your feedback. I am glad you find FM Bench Analyzer so valuable. Although we don’t have a free trial ready yet you can still take advantage of our 30 day money back guarantee.
While the whole FM Bench package including Analyzer is designed as a complete solution for optimizing large database applications, FM Bench Detective is useful even alone in situations when you know what you want to optimize as it provides a simple way to precisely benchmark scripts, calculations, and even layouts.
In fact, several developers have already reported amazing results from using just Detective.
Even the “marvelous optimizations” I wrote about on my blog were achieved solely with the help of FM Bench Detective and I only used FM Bench Analyzer to demonstrate the results.