ViewVC Help
View Directory | Revision Log | View Changeset | Root Listing
root/Oni2/Validate External Links
Revision 1177 - Directory Listing - [select for diffs]
Modified Fri Jan 13 22:26:56 2023 UTC (2 years, 8 months ago) by iritscen
Diff to previous 1175
ValExtLinks now skips URLs that aren't HTTP(S) protocol. Added some error-checking on line parsing. Added my email address.
Revision 1175 - Directory Listing - [select for diffs]
Modified Tue Aug 23 14:15:48 2022 UTC (3 years, 1 month ago) by iritscen
Diff to previous 1160
ValExtLinks: Added audit feature which tells the user if there are items in the exception list which are no longer present on the wiki or no longer return the given error code.
Revision 1160 - Directory Listing - [select for diffs]
Modified Sun Aug 15 14:20:21 2021 UTC (4 years, 1 month ago) by iritscen
Diff to previous 1159
ValExtLinks: Added some entries to the lists of known file and page suffixes.
Revision 1159 - Directory Listing - [select for diffs]
Modified Sun Jun 27 18:35:28 2021 UTC (4 years, 3 months ago) by iritscen
Diff to previous 1158
ValExtLinks: Updated sample files which go along with the last update to the script.
Revision 1158 - Directory Listing - [select for diffs]
Modified Sun Jun 13 20:50:43 2021 UTC (4 years, 3 months ago) by iritscen
Diff to previous 1157
ValExtLinks: Added argument to 'curl' that prevents some sites from rejecting it. Val now skips archive.is links too when skipping archive.org links.
Revision 1157 - Directory Listing - [select for diffs]
Modified Sun May 9 21:53:48 2021 UTC (4 years, 5 months ago) by iritscen
Diff to previous 1150
ValExtLinks: Make sure that bad YT links count as NG. Various tweaks to project organization.
Revision 1150 - Directory Listing - [select for diffs]
Modified Tue Feb 23 22:35:00 2021 UTC (4 years, 7 months ago) by iritscen
Diff to previous 1149
ValExtLinks: Added files to version control that I used for research when learning how to detect bad YouTube links. Updated to-do list.
Revision 1149 - Directory Listing - [select for diffs]
Modified Sun Feb 7 22:36:56 2021 UTC (4 years, 8 months ago) by iritscen
Diff to previous 1148
ValExtLinks: The messages about skipping URLs now show the wiki page's namespace. Added 504 to known response codes.
Revision 1148 - Directory Listing - [select for diffs]
Modified Thu Feb 4 23:15:20 2021 UTC (4 years, 8 months ago) by iritscen
Diff to previous 1147
ValExtLinks: Val can now recognize bad YouTube links (no thanks to YouTube). Fixed some math errors. Added error 429 to known codes.
Revision 1147 - Directory Listing - [select for diffs]
Modified Tue Feb 2 20:10:39 2021 UTC (4 years, 8 months ago) by iritscen
Diff to previous 1146
ValExtLinks: Changed --suggest-snapshots to --suggest-snapshots-ng and added --suggest-snapshots-ok for getting snapshot URLs for all good links. This can be used to confirm that sites are backed up in case they die in the future, but note that this argument will take hours to run due to the API rate limit. Added awareness of API rate limit so Archive.org will not start blocking script.
Revision 1146 - Directory Listing - [select for diffs]
Modified Sun Nov 1 18:55:05 2020 UTC (4 years, 11 months ago) by iritscen
Diff to previous 1145
ValExtLinks: Replace all occurrences of HTML-encoded '&'s in exception URL, not just the first.
Revision 1145 - Directory Listing - [select for diffs]
Modified Sun Oct 11 14:58:59 2020 UTC (5 years ago) by iritscen
Diff to previous 1144
ValExtLinks: Added .do to recognized page suffixes.
Revision 1144 - Directory Listing - [select for diffs]
Modified Sun Sep 6 20:51:22 2020 UTC (5 years, 1 month ago) by iritscen
Diff to previous 1143
ValExtLinks: Changed --skip-archive-links argument to --check-archive-links because the default should be to skip them. Val now uploads all three formats of its report, and links to the RTF and TXT versions from the HTML one. Val can also now tell whether each upload succeeded. A report with no link issues will print a placeholder message in that section of the report. Fixed a bug where Val thought a link should be an interwiki link when it was really a link to an archive.org snapshot from said wiki.
Revision 1143 - Directory Listing - [select for diffs]
Modified Fri Sep 4 13:11:57 2020 UTC (5 years, 1 month ago) by iritscen
Diff to previous 1142
Committing sample files to go along with last committed version of ValExtLinks.
Revision 1142 - Directory Listing - [select for diffs]
Modified Fri Sep 4 03:07:08 2020 UTC (5 years, 1 month ago) by iritscen
Diff to previous 1141
Val now tries each URL three times. This has proven more effective than giving Val a long timeout and trying each URL once. The summary report has been refined a bit; the most notable change is that the final number and breakdown of link issues leaves out the excepted links. Also stopped Val from getting confused by HTML-encoded '&'s in the exceptions list.
Revision 1141 - Directory Listing - [select for diffs]
Modified Fri Sep 4 02:54:30 2020 UTC (5 years, 1 month ago) by iritscen
Diff to previous 1139
Committing the changes to Val which I meant to commit over a week ago. I committed everything but the updated script itself. See last Val commit message for list of changes.
Revision 1139 - Directory Listing - [select for diffs]
Modified Wed Aug 26 18:33:35 2020 UTC (5 years, 1 month ago) by iritscen
Diff to previous 1138
ValExtLinks: The 'I Just Broke All The Diff-Gazing' Commit. Revised file structure of the project and added the MIT license. Revised all documentation. HTTP and curl codes are now wiki-formatted and hosted online. In Val itself, I made these changes: Added --timeout argument to change default URL query timeout of 10 seconds. Revised the report header to be easier to read. Less advice is printed directly in the report and more is online at the location linked to under the Legend section.
Revision 1138 - Directory Listing - [select for diffs]
Modified Thu Jul 23 21:15:18 2020 UTC (5 years, 2 months ago) by iritscen
Diff to previous 1137
Adding my email address to some of my projects.
Revision 1137 - Directory Listing - [select for diffs]
Modified Tue Jul 21 14:16:54 2020 UTC (5 years, 2 months ago) by iritscen
Diff to previous 1136
ValExtLinks: Added '.full' as a recognized page suffix.
Revision 1136 - Directory Listing - [select for diffs]
Modified Mon Jul 20 15:58:39 2020 UTC (5 years, 2 months ago) by iritscen
Diff to previous 1135
ValExtLinks now reads its exceptions list from a wiki page instead of a text file that only I can change.
Revision 1135 - Directory Listing - [select for diffs]
Modified Sun Jul 12 23:57:00 2020 UTC (5 years, 2 months ago) by iritscen
Diff to previous 1127
Added option to not validate archive.org URLs, as those are unlikely to go bad, and we have an increasing number of them. Val now reports trivial redirect settings in Config section.
Revision 1127 - Directory Listing - [select for diffs]
Modified Sat Mar 28 02:08:29 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1125
Val now counts redirects from youtu.be to youtube.com as OK links.  These links will be reported on if the argument --show-yt-redirects is used.  Renamed --show-https-upgrade to --show-https-upgrades for consistency.  Also sorted the file and page suffix arrays and added some more items to them.  Now handling status codes 400, 418, 502 and 530.  Fixed incorrect nbsps in HTML report.  Val is no longer confused by URLs ending in '(' or ')', or which contain a '%' towards the end.
Revision 1125 - Directory Listing - [select for diffs]
Modified Wed Mar 25 21:50:30 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1124
Val no longer prints the result codes for IW and EI links, since that doesn't really make sense. Fixed a spacing issue in HTML report.
Revision 1124 - Directory Listing - [select for diffs]
Modified Wed Mar 25 01:59:27 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1123
Val now removes the annoying ':80' in many Archive links.
Revision 1123 - Directory Listing - [select for diffs]
Modified Sat Mar 21 22:08:35 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1122
Fixed bug in Val that was causing dozens of 403 errors to be returned unnecessarily. Polished report wording and messages a little.
Revision 1122 - Directory Listing - [select for diffs]
Modified Fri Mar 20 22:13:48 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1121
Val now links to wiki pages using HTTPS instead of HTTP. Fixed code that exempts minor forms of redirects from being listed. New arguments --show-added-slashes and --show-https-upgrade allow one to turn off these exemptions. Reworked summary section extensively to be more readable.
Revision 1121 - Directory Listing - [select for diffs]
Modified Wed Mar 18 18:36:54 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1120
Changing this copy of the Val exceptions file to a sample version so I don't have to keep updating it in version control. The live version is at http://iritscen.oni2.net/val/exceptions.txt.
Revision 1120 - Directory Listing - [select for diffs]
Modified Wed Mar 18 17:08:59 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1119
Val's reports now print section headers for the init/config stage and for the link results themselves.
Revision 1119 - Directory Listing - [select for diffs]
Modified Wed Mar 18 00:24:42 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1118
Properly fixed Val's parsing of Archive API responses this time. Added a little space between each link result, making report much easier to read.
Revision 1118 - Directory Listing - [select for diffs]
Modified Tue Mar 17 16:07:35 2020 UTC (5 years, 6 months ago) by iritscen
Diff to previous 1075
Fixed ValExtLinks' reading of Archive API replies. Fix for reading links that happen to have a shebang in them. Now knows how to handle NULL namespace links. Now prints elapsed time.
Revision 1075 - Directory Listing - [select for diffs]
Modified Fri Oct 6 02:02:16 2017 UTC (8 years ago) by iritscen
Diff to previous 1070
ValExtLinks: Fixed bug in interwiki link suggestions. Corrected documentation error.
Revision 1070 - Directory Listing - [select for diffs]
Modified Tue Oct 3 03:01:32 2017 UTC (8 years ago) by iritscen
Diff to previous 1069
ValExtLinks improvements:
- Now advises reader of external internal links.
- The exceptions file now allows finer-grained exemption of a URL by matching to the specific page that contains it instead of exempting all occurrences of that URL (but the '*' wildcard will match all containing pages). Currently you can only list a URL once, however.
- The exceptions file now allows external internal and potential intrawiki links to be exempted from the report.
- The path to Google Chrome (for taking screenshots) is now external to the script, supplied as an argument after "--take-screenshots".
- All of OniGalore's interwiki shortcuts are now recognized.
- Protection against failed retrieval of redirect URL.
- Better recognition of unimportant redirects (http->https, added ending slash).
Revision 1069 - Directory Listing - [select for diffs]
Modified Wed Aug 2 04:26:48 2017 UTC (8 years, 2 months ago) by iritscen
Diff to previous 1068
ValExtLinks: IW links now reported as separate category from OK links. RD links that are just redirecting from http:// to https:// are now regarded as OK.
Revision 1068 - Directory Listing - [select for diffs]
Modified Tue Aug 1 17:10:48 2017 UTC (8 years, 2 months ago) by iritscen
Diff to previous 1067
Updating http_codes.txt in line with what Val now understands.
Revision 1067 - Directory Listing - [select for diffs]
Modified Tue Aug 1 17:09:42 2017 UTC (8 years, 2 months ago) by iritscen
Diff to previous 1066
Val now understands HTTP redirect responses and will report the URL we're redirected to. Also now tallies IW links.
Revision 1066 - Directory Listing - [select for diffs]
Modified Tue Aug 1 14:30:24 2017 UTC (8 years, 2 months ago) by iritscen
Diff to previous 1064
Updating Val to new location of files on oni2.net and slight changes to Archive API.
Revision 1064 - Directory Listing - [select for diffs]
Added Sun Jul 2 21:50:22 2017 UTC (8 years, 3 months ago) by iritscen
Committing my wiki link validation script, as it is reasonably mature now.

Convenience Links