GFL_SORTTYPE

[Bob Hansen]

I'm happy you are content, but this is not over for me. This has become a personal issue now. We are running the same scripts, yours and mine and getting different results. The difference in our testing is the actual files we are trying to sort. I need the same real files to test.

Can you send me four real files that you download for me to use here? I thought about this last night and realized what you mentioned above, they all have the same date/time stamp because of how they are created. You will not be able to sort by date properly with 3/4, unless you change the date to the date of the email.

Since they are emails, I think what you need to do is download the files into a directory, put them into a loop, open them up, use RegEx in the email header to extract the Send Date/Time from the header. Then use that info to change the date/time of the file. I don't think Macro Scheduler has a command to do that (SUGGESTION FOR NEW COMMANDS), but there are some "Touch" utilities that can do that. If you cannot Touch the date, you could rename them, including the date/time, and then parse the file name to sort by the position of the date/time in the file name.

As I noted above, you may be done, but I need to see and understand and correct this issue. I hope you will humor me and provide me with real files (Make them impersonal, since I will be looking inside).

Good luck, and someday in the future you may receive a surprise message!

Thanks,

Bob

[Bob Hansen]

Can you also let me know how you are saving the email files. If using Macro Scheduler code, please provide that. You said you are using POP3 connection, using which email client? After connected, how do you select the files and save them to disk. In a group, one at a time?
I am sure the problem is in the files that are being sorted, I believe GFL is working properly.

Again, this has now become a personal issue for me, and I shall prevail!

Thanks again.

[mightycpa]

Hi Bob,

You must have missed my long-winded yet sparsely detailed explanation about how I pointed your script to a different folder, which contains real life files of all shapes, sizes and origin dates. Like this:

Code: Select all

Let>GFL_SORTTYPE=
GetFileList>C:\Users\mightycpa\Documents\*.txt,files
MessageModal>files
... etc.

You couldn't ask for a better test than that one. I got the same exact behavior. You are headed down the wrong path; the answer does not lie in the files, but rather, the filesystem. I am convinced of that.

I will try the other machine sometime this weekend and see if it behaves better. Stay tuned!

[Bob Hansen]

I just saved POP3 emails from Thunderbird Email client. I selected them, did Save As to C:\Temp folder. They all saved with a format of Subject - Sender - Sent Date/Time.eml
My file example:
Article V Process - Sendername ([email protected]) - 2021-03-06 1131.eml

This means we do not need to use RegEx in the email header to extract the Date/Time info!
They all show the same Date/Time in Windows Explorer, does not show seconds. But the file name does show time in the last four digits. So, if we take the Date/Time at the end and rename the file putting that at the front,
2021-03-06 1131 - Article V Process - Sendername ([email protected]).eml
Then you can sort them by NAME (not date) Asc.
Then change back to original name. And you will have the files listed by Date/Time.

Or, you can use the Date/Time in the file name with a Touch utility and change the dates of the files to the dates in the file name. Then, in the future you will be able to sort by Name, by Date, by size.

I would prefer to use the second, Touch method, for permanent correct values.
It may sound like a lot of steps, but Macro Scheduler can do these tasks in seconds.
=================
Now your files may have a different format, because you may be using different tools to save the email files.

[Bob Hansen]

You noted:

You must have missed my long-winded yet sparsely detailed explanation about how I pointed your script to a different folder, which contains real life files of all shapes, sizes and origin dates. Like this:

I did see that, but had no access to your actual files. So I used my Testx,txt files to test GFL, and found it working fine. You later replied with files named MSGx.txt. I seemed like we were performing the same tests with different results.

But I have still been unable to see GFL fail, not with test.txt files, and not with real.eml files, except sorting by date does not work with identical dates. That is no surprise. So the Sensible Solution will be to fix the file dates to match the email dates.

Still smiling here, I love a challenge.

[mightycpa]

But that's my point, Bob. Even with a population of files having different dates spanning years, the date sort did nothing for me. For whatever reason, the variable doesn't work for me.

ON THIS MACHINE. However, on my Windows 7 machine, using MS15, it actually does work as billed. Ditto on the Server2008 Machine, MS15.

So now the only question is: Is it Windows Enterprise? Or is it MS14.3.03? The only way to know:

Code: Select all

// COMPILE_OPTS|C:\Users\GeorgeAlvarez\Documents\Macro Scheduler 14\Samples\get move ready area\gfl_14.exe||CONSOLE=0|INCLUDES=1||RUNTIMES=1|BMPS=1
//Set IGNORESPACES to 1 to force script interpreter to ignore spaces.
//If using IGNORESPACES quote strings in {" ... "}
//Let>IGNORESPACES=1
//FILE PROCESSING
//DELETE *BODY*.TXT
//ONLY THING THAT REMAINS IS MSG?.TXT

Let>INPUT_BROWSE=2
Input>v_filespec,Type in path,c:\
Let>GFL_SORTTYPE=
GetFileList>%v_filespec%\*.*,files
MessageModal>files
Let>GFL_SORTTYPE=0
GetFileList>%v_filespec%\*.*,files
MessageModal>files
Let>GFL_SORTTYPE=1
GetFileList>%v_filespec%\*.*,files
MessageModal>files
Let>GFL_SORTTYPE=2
GetFileList>%v_filespec%\*.*,files
MessageModal>files
Let>GFL_SORTTYPE=3
GetFileList>%v_filespec%\*.*,files
MessageModal>files
Let>GFL_SORTTYPE=4
GetFileList>%v_filespec%\*.*,files
MessageModal>files
Let>GFL_SORTTYPE=5
GetFileList>%v_filespec%\*.*,files
MessageModal>files
Let>GFL_SORTTYPE=6
GetFileList>%v_filespec%\*.*,files
MessageModal>files

Compile the new code and test. The answer surprised me.

https://www.filedropper.com/gfl14

http://www.filedropper.com/gfl15

[Bob Hansen]

I have been busy working to fix the same date/time of the files to allow Sorting to work by Sent Date. I will take a look at what you did, later today.
You might want to consider this code, regardless of any system, because the SaveAs command will make all the files have the same Date/Time. This routine will take the Date/Time from the file name, and actually change the date of the Saved Files to the Date/Time in the file name.

Code: Select all

//Designed for file name format: xxxxxxxx - YYYY-MM-DD HHMM.eml
//Loop to run for every file in FileList

Let>vLoopCount=0
Label:Loop Files

Let>VAREXPLICIT=0
//Get Next file from File List for next line vChangeFile
Let>vChangeFile=C:\TEMP\TestMail - 2021-03-06 1348-3.eml

//Process Date
Let>vPattern=[0-9]{4}-[0-9]{2}-[0-9]{2}
RegEx>vPattern,vChangeFile,0,vDateMatch,vDateCount,,,
Let>vDate=vDateMatch_1

//Process Time
//Hr-Min may have a trailing "-#" that will be dropped
Let>vPattern=[0-9]{4}-?[0-9]?\.eml
RegEx>vPattern,vChangeFile,0,vTimeMatch,vTimeCount,,,
MidStr>vTimeMatch_1,1,6,vTime
MidStr>%vTime%,1,2,vHr
MidStr>%vTime%,3,2,vMin

//Touch program needs time values in quotes
Let>vPgmChangeFile="%vChangeFile%"
Let>vDate="%vDate%"
Let>vTime="%vHr%:%vMin%:00"

//Get original file date/time
FileDate>vChangeFile,vOrgDate
FileTime>vChangeFile,vOrgTime

//Change file/date to actual Send values
//This calls the "FS TOUCH", free utility with GUI and command lines 
//Downloaded on 4/9/2014 from: http://www.funduc.com/fstouch.htm

RunProgram>C:\Program Files\FSTouch\fsTouch64.exe /p%vPgmChangeFile% /d%vDate% /t%vTime%
Wait>1

//Get modified file date/time
FileDate>vChangeFile,vNewDate
FileTime>vChangeFile,vNewTime
MessageModal>%vOrgDate%, %vOrgTime%  %CRLF%%vNewDate%, %vNewTime%  

//Let>vLoopCount=%vLoopCount%+1
//If vLoopCount = File Count, Exit Loop, Else Repeat Loop
Label>EndLoop

I left the final MessageModal there so you can initially Step thru the code and see the final differences. You should comment out the last line in your final code.

I did not create the looping code around this, but the structure should be easy to implement if you have done loops before.

I checked the web site for FS TOUCH, and it is still available. Provides both 32 and 64 bit versions. You may see a few sorted records off a line or two if you have more with the same date/hr/minute because the seconds are not in the file name. I did not see any files like that on my system, but set up the RegEx to allow for that. But FSTOUCH does allow you to change the seconds also, if you find that becomes a problem. I think it will be rare.

So, I will look into your last message, but do we now agree that GFL is working as expected? The problem is somewhere else. This routine will eliminate most sorting saved emails by Date/Time for any program, not just MacroScheduler.

Good luck, good working with you. I am looking into the next chapter from your latest message.

[Bob Hansen]

OK, back again, that was quick.
I did not compile the code.
But I ran the code against my MSG#.txt test files.
All of the MessageModules showed me the correct results, as best as I could see. Looks like GFL is working fine.
I then used FS TOUCH to change the dates on all four files to the same date/time, same hr/min/sec values. This time everything looked good again, except the sort by Date. Both results for 1/2 were identical. It was a random order.

I then used FSTOUCH to change the minutes only on the four files. This time the Date sorting was correct, both ascending and the reverse descending were what should be expected.

So, from this, I conclude again, that GFL works as expected.
But sorting files with identical date and time will probably be erratic. This is probably of most programs unless they get down to the "seconds" level.

So I then changed the hr/min to be the same, and just changed the "seconds". When I ran the files using your latest code,they all sorted properly, so this shows that Macro Scheduler can, and does, sort dates down to the "seconds" level.

Which brings me back to how the files are saved. I don't have access to your files, but I suspect when you do SaveAs, many/most/all will get the Hr/Min/Sec of the time the command was executed assigned as the file Date/Time that we see in Explorer, and that sorting programs see internally.

So, the last thing I can think of, to clear up any more questions by me, is if you can do a File SaveAs to a group of emails, and then send the four files to me so I can do some final testing using the same files that you are testing with.

Thanks, good luck with comparing your OS possible problems.

[mightycpa]

Hold on.

You ran both GFL14 and GFL15, and they both worked correctly for you?

Because I ran GFL14, and it did not sort on either machine. Only GFL15 sorted properly on both machines.

Here's a movie: https://app.vidjack.com/share/1074

Yes, it would prove nothing if you compiled the code. We already know it works on your machine.

[Bob Hansen]

Great job on the video. It was easy to see the difference without having to read. All I can say now is WOW! You have a problem! Have you thought about contacting the Macro Scheduler Forum for help? (Sorry, I could not help myself, )

No, I did not run GFL15. I have been retired almost 10 years, and see no need to keep upgrading. But I am using 14.7, so, not many differences.

I still think there is something about your files that are causing this. And I have not had access to them, so there is no way I duplicate the tests properly. I have one major variable that is not the same, the files that are being sorted.

I have just sent you a PM with my email address, hoping you will send them as an attachment, or will give me your email address so I can set up a DropBox destination for an upload.

I wonder if there is some type of conflict because you have both 14/15 on the same system? Could there be some type of registry setting that is being read/or not?
Again, you seem to have a solution that works for you, so maybe you don't need to put in any more time for an answer. But as I said, I hate to let this go without understanding the cause.

No one else has jumped in from the forum, maybe Marcus needs to be asked directly?

So, not much more I can do without your files. I may get them, and know no more, but at least I will have eliminated all of the possibilities that I could think of. There is no way I can assist on the systems you are using, I cannot duplicate multiple OS here.

Will be watching for a reply to my PM, thanks.

[mightycpa]

Well, actually, I don't have MS installed on that machine where I took the video. Those are compiled programs only.

If this is Post_0 (posts sorted in descending order, don't you know ), then in Post_5, at the very bottom, you'll see two links, one for the compiled GFL14 (ie, using MS14), the other for compiled GFL15 (using MS15).

If you download and run each of them, I believe that you will be able to recreate what you saw in the video.

Give it a try, let me know what happens. Those two are exactly what I ran in the video, and you can use whatever folder you want when you run it. The only thing I haven't done yet is run GFL15 with the files I get when I fetch emails. I remain convinced that doesn't matter. We'll see soon enough.

[Bob Hansen]

Thanks, that was a great learning class. Here are my comments:
1. I did try the links on the day you posted, but it looked like a place to upload files. But thanks to your last email, I was able to look closer and download. Thanks for the new site to add to my toolbox.
2. AVG on my system thought GFL14 might be bad, so moved it to special area, might be a few hours to authenticate. I said go ahead anyway. (I think MS15 allows you to add a certificate to prevent this?).
3. I ran the file, stopped after screen1, and checked and saw your emails in my C:\Temp\ folder, well done. They have the same naming format with date info at the end, so my earlier Touch routine should work well.
4. I finished the routine and it looked like NO sorting! That may be good news. Because I had no problems with my files on my system. You must be happy. You were able to let me see a duplicated problem with your files. I do agree, you seem to have a problem.
5. Good news: So now I can test your files with my MS.
5. Bad news: I probably cannot get to this until later today, so there will be few hours delay. But that will also give me some time to plan the next attack.

Stay tuned, I am feeling good about this.

[mightycpa]

I'm not clear if you tried my GFL15 or not, but as of right now I have two solutions:

ArraySort for 14
Upgrade to 15

[Bob Hansen]

Did not leave yet, but I spoke too soon in last message.
The files I saw with GFL14 were my own files, from earlier tests. And these were files that all had the samed time (hr/min/sec), so that is why they did not sort in your exe file. There were no new files copied onto my system, so I do not have a copy of your files to test with.
Only got the GFL14 at first because I don't have 15, but will download and try that later also.
Gotta go......

[mightycpa]

You don't need my files. You can use any files on your system, including files with identical content and identical dates, or completely different files and dates. In the movie, I used the files from MS Sample folder.

My GFL14.exe ad GFL15.exe will yield different results every time.