                              YTConfig v0.50b
                              ===============
             YTCONFIG.TXT - Copyright (c) 2019, Fred C. Macall
   

Overview

YTConfig is a DOS program for extracting a signature descrambling rule or
key from a saved copy of a YouTube video player JavaScript file.
YTCONFIG.EXE joined the YTCrack package beginning with little published
version 0.30b .  YTConfig helps YTCrack work with YouTube Web pages
carrying invalid or scrambled signatures!  The ones that YTCrack, by
itself, couldn't handle for much of 2013 and 2014.  This document covers
YTConfig's side of its interfaces with YTCrack.  YTCRACK.TXT covers
YTCrack's side of these interfaces.

YTCrack's report file document informs its user of a URL and version name,
for the YouTube player or base.js JavaScript file, that may need
consulting.  This file and its version name are expected to be updated as
often as daily.  YTCrack and YTConfig take the version name to be the last
8 characters of the <version> part of the file's full URL.  We'll call the
version name:  <version8> , to distinguish it from the longer <version>
field in the file's full URL.  The user is expected to receive the file
into a local file named <version8>.js , when necessary.  (Though, the .js
part of that is optional.)  Then, this local file is to be named as the
input file in a YTConfig run.  This process becomes necessary when YTCrack
reports:
nn s= . . . style signature(s) found.
followed by:
0 of these corrected using version based rule.

YTConfig finds the signature descrambling rule or key embedded in its input
file and conveys that result via a one line DOS .BAT file.  This file is
expected to contain a DOS
SET <version8>=<descramblingrule>
command that will make YTConfig's result available to YTCrack via the DOS
Environment.  After this .BAT file has been run, a rerun of a YTCrack run
that had been unsuccessful, due to the presence of unhandled invalid or
scrambled signature(s), is expected to succeed.  And, report:
nn of these corrected using version based rule.

Once YTConfig has been run for a given <version8> , it is sufficient to
rerun the .BAT file if its <version8>=<descramblingrule> should get lost
from the DOS Environment for any reason.

YTConfig uses no configuration or helper file or program and is intended to
run on 8086/8088 based, and all later, IBM PC compatible PCs.
It only needs a modest amount of DOS memory.  But, it may run for a few
seconds or minutes on an older (read, slower) PC.  It requires only
DOS v3.x, or later.


History

Beginning in July 2014, YouTube's html5player.js files started exhibiting
a new class of JavaScript wrappers for the native JavaScript functions
actually used in the signature descrambling rule that YTConfig seeks to
learn.  This change can only be understood as an effort to introduce a new
level of unnecessary or obfuscational complexity in html5player.js .
YTConfig copes with this change, beginning with version 0.32b, as explained
in the next section.  

YouTube is continually "fuzzing" or changing the variable names in their
html5player.js file.  The week of 16 November 2014, the class of JavaScript
wrappers introduced in the paragraph just above was named ar .
This choice exposed a mistake in YTConfig v0.33b .  It couldn't find the
sequence:  var ar={ in that week's html5player.js files!  Even though the
sequence was waiting to be found, in plain sight.  Names for that class of
wrappers such as v, a, r, and va would have exposed the mistake, as well.
YTConfig version 0.34b contains a correction for this mistake.

The version and date labeling in YTConfig versions 0.35b, 0.36b, 0.37b, and
0.38b have been updated to match the accompanying YTCrack version.

On or about 9 November 2015, YTConfig versions 0.38b and before became
incapable of extracting a descrambling key, for &s= . . . style signatures,
from YouTube's video player.  That turned out to be caused by a change in
the style of the player's signature descrambling function definition.
It changed from:
function fn( . . .
style to:
var fn=function( . . .
style.  That trouble might have gone unnoticed at the time.
Because, at the same time, YTCrack versions 0.38b and before quit
recognizing and displaying a URL for YouTube's video player, at the bottom
of their report.  That turned out to be caused by a change of the player's
name, from html5player to player or base.  Also, even if YTConfig had
continued working, YTCrack wouldn't have been able to use its result
without having found the applicable version string.

After finding fn in its first search, YTConfig version 0.40b now looks for
var fn=function( . . . }
instead of
function fn( . . . }

On or about 23 November 2015, YouTube added some optional white-space, in
the form of two extra new-line sequences, to their video player.
These extra new-line sequences appeared within the definition of the class
of descrambling operators that YTConfig seeks.  As a result,
YTConfig v0.40b got confused and failed.

Beginning with version 0.41b, YTConfig has been extensively revised to
ignore any and all extra or optional white-space that may be found in the
player or base.js file that it processes.  To explain this, the following
section of this document has been extensively revised, as well!

On or about 18 December 2015, YouTube included the definition of its
function for descrambling scrambled or invalid &s= . . . style signatures
in a compound var statement containing several function definitions.
So, var no longer appears immediately ahead of fn=function( . . . } for the
descrambling function definition.  This new obfuscation keeps
YTConfig v0.41b from working.

After finding fn in its first search, YTConfig version 0.42b now looks for
fn=function( . . . }
instead of
var fn=function( . . . }

On or about 20 January 2016 and 4 February 2016, YTConfig's searches for
fn=function( . . . }
failed.  Because YouTube's fuzzing of their video player script's
identifiers caused fn's actual value to match substring(s) of longer
identifiers.  For example, on 20 January 2016, YTConfig was searching for:
or=function( . . .
That search found:
. . . error=function( . . .
instead.  This kind of issue may be seen to be due to omission(s) in
YTConfig v0.42b.

YTConfig version 0.43b has been extended to check the character ahead of a
tentative finding of:
fn=function( . . . }
If it is in the set of A - Z , a - z , _ , $ , and 0 - 9 , or the
characters allowed in JavaScript identifiers, such a tentative finding gets
ignored and YTConfig continues its search.  YTConfig's provision(s) for
recovering from such false tentative matches, in its search, have been
augmented, as well.

The version and date labeling in YTConfig version 0.44b have been updated
to match the accompanying YTCrack version.

On or about 31 January 2017, JavaScript code in YouTube's player or base.js
files, for handling &sig= . . . and &s= . . . style signatures, changed.
This change kept YTConfig version 0.44b and all before from working.

YTConfig version 0.45b recognizes the new code sequence, as described
below.

On or about 14 March 2017, apparent refactoring or restructuring in
YouTube's player or base.js file resulted in it outgrowing the 8 KB input
file buffer that YTConfig v0.45b used for its searches.

In YTConfig version 0.46b, that input file buffer has been enlarged to a
little under 32 KB.

On or about 28 April 2017, YouTube changed the structure of the path in the
URL for their player or base.js file.  They moved the URL's language field
(for example:  en_US) into its own directory in the path.  As a result,
YTCrack and (sometimes) YTConfig, version 0.46b, started reporting a player
or base.js file "version" different from what they had been reporting.
This change could be worked-around with some difficulty.

Both YTCrack and YTConfig, version 0.47b, have been tweaked so as to report
the player or base.js file's version as they had, before.

On or about 13 February 2018, more apparent refactoring or restructuring in
YouTube's player or base.js file resulted in YTConfig failing to recognize
the signature related expression it seeks.  That result was due to the :
that had been prefixing the expression being replaced by a ); sequence.

In YTConfig version 0.48b, the needed signature related expression will be
found when prefixed by either : or ); .

On or about 6 Novenber 2018, more apparent refactoring or restructuring in
YouTube's player or base.js file again resulted in YTConfig failing to
recognize the signature related expression it seeks.  The call to the
top level signature descrambling function has been surrounded by apparent
casts.

YTConfig version 0.49b handles the new complexity and still works with the
cleaner preceding structure.

On or about 15 January 2019, another change or simplification was made in
the same area of YouTube's player or base.js file.  This change tripped up
YTConfig again.  YTConfig version 0.50b has been extended to correctly
handle all three of the recently seen versions of the player or base.js
file's top level signature descrambling call.


How It Works

YTConfig starts off by checking for a pair of command line parameters that
do not match each other.  Finding anything else, it displays its Usage
message and terminates.  For the record, this message reads:

Usage:  YTCONFIG <input file name> <output file name>

Notes:  <input file name> specifies a YouTube
        player-<version>/base.js file.
  And:  <output file name> specifies a .BAT file to be written.
        The two file names must differ.

YTConfig then opens both of its given files.  And, reads all the way
through its input file looking for JavaScript statement(s) of the form:

var<white-space><wildtxtz><sep><wildtxt0>)

Where:  <sep> is : (before version 0.48b) or ); (since).
        <wildtxtz> is any text that doesn't include <sep> nor var .
        <wildtxt0> is any text that doesn't include ) nor var .

YTConfig makes a white-space-free copy of the <sep><wildtxt0>) portion of
each such statement and then checks that for a phrase of the form:

<sep><wildtxt1>.s&&<wildtxt2>,<wildtxt3>(<wildtxt1>.s)

Where:  <wildtxt1> is any text that doesn't include .s&& , <comma> , 
        nor .s) .  The two instances of <wildtxt1> must be identical.
        (Also, it is known to be free of white-space , ) , and var .)
        <wildtxt2> is any text that doesn't include <comma> , nor .s) .
        (Also, it is known to be free of white-space , ) , and var .)
        <wildtxt3> is any text that doesn't include .s) .
        (Also, it is known to be free of white-space , ) , and var .)

YTConfig then reports on what it has found.

If it has found at least one JavaScript phrase matching the above
description, it rewinds its input file and looks for statement(s) of the
form:

var<wildtxt4><wildtxt3><wildtxt5>var

Where:  <wildtxt3> is from the first phrase found above.
        (In the "History" section of this document, this sequence is
        called:  fn .)
        <wildtxt4> is anything that doesn't contain <wildtxt3> nor var .
        Beginning with version 0.43b , YTCrack checks to see if the last
        character of <wildtxt4> might be a JavaScript identifier character.
        I.e.:  A - Z , a - z , _ , $ , or 0 - 9 .  If so, such a tentative
        match gets discarded and the search continues.
        <wildtxt5> is anything that doesn't contain var .

When it finds such a statement, YTConfig makes a white-space-free copy of
everything from <wildtxt3> through the end of <wildtxt5> .  Then, it checks
for a statement of the form:

<wildtxt3>=function(<wildtxt6>}

Where:  <wildtxt3> is the same old function name we are seeking.
        <wildtxt6> is anything that doesn't include } .
        (Also, it is known to be free of white-space and var .)
 
Such a statement is expected to be a JavaScript definition for the function
that the video player uses for descrambling &s= . . . style signatures.
This function definition was expected to consist of JavaScript code for a
sequence of the Reverse, Slice(nn), and sWap(nn) operations described in
the How it Works section of YTCRACK.TXT .  The main complication was that
the code for sWap(nn) took one form when it appeared only once and another
form when it appeared multiple times, in the function definition.
Since the middle of July 2014, this function has been defined in terms of
calls to members of a class of wrappers for the JavaScript Reverse,
Slice(nn), and sWap(nn) operator functions.

Having found such a function definition, which is taken to end at the }
shown above, YTConfig displays it and parses its <wildtxt6> portion, in a
loop over its text containing the following body:

        Look for period and left parenthesis, whichever is next.

        If period is found first or alone, expect one of the sequences:
                .join(       >>  Successful conclusion.
                .length]     >>  Look for [nn] and emit wnn .
                .reverse(    >>  Emit r .
                .slice(      >>  Look for nn) and emit snn .
                .split(      >>  Continue.
                .<other>     >>  Non-successful conclusion.

        If left parenthesis is found first or alone, expect:

                (<name>,nn)  >>  Emit wnn .

        Otherwise, error conclusion.

Before version 0.32b, the "non-successful conclusion" path shown above
always was taken to be an error conclusion.  Since version 0.32b, checks
are made to decide if one of the following cases has been encountered:

        ;<name>=<class>.<member>(

               ;<class>.<member>(

If so, and nothing has been emitted, yet, YTConfig rewinds the input file
again and looks for the sequence(s) of the form:

var<white-space><wildtxt7>{

Where:  <wildtxt7> is anything that doesn't contain { nor var .

When it finds such a sequence, YTConfig makes a white-space-free copy of it
and checks that copy for:

var<class>={

Where:  <class> is the name found as shown above.
      
When it finds such a sequence, YTConfig returns to the
var<white-space><wildtxt7>{
sequence just identified in the raw file buffer and looks further for a
sequence of the form:

var<white-space><wildtxt7>{<wildtxt8>}<wildtxt9>}

Where:  <wildtxt8> is anything that doesn't include a pair of
        right curly brackets unseparated by a left curly bracket.
        <wildtxt9> is anything that doesn't include { nor } .

If and when it finds such a sequence, YTConfig makes a white-space-free
copy of it and checks that copy for:

var<class>={<wildtxt10>}}

Where:  <wildtxt10> is anything that doesn't include }} .
        (Also, of course, it is known to be free of white-space .)

Such a statement is expected to define a class of wrappers for the
JavaScript descrambling operation sequences that the descrambling function
used to use directly.  You'll see that a little twist here is that either
of the JavaScript functions .slice( ) or .splice( ) may be invoked for the
Slice(nn) descrambling operation.  Having found such a statement, YTConfig
displays it and parses it's <wildtxt10> portion, in a loop over its text
containing the following body:

        Locate the character following the ={ shown above, the first time.
        Or, locate the character following any (next) appearance of }, .
        The sequences expected here are:

        <member>:function(<arguments>){<definition>}

        Where:  <definition> is expected to contain one of the sequences:

                .reverse(  >>  Register <member> as wrapper for Reverse.
                .slice(    >>  Register <member> as wrapper for Slice(nn).
                .splice(   >>  Register <member> as wrapper for Slice(nn).
                .length]   >>  Register <member> as wrapper for sWap(nn).

        Otherwise, error conclusion.

Here, a successful conclusion comes after another appearance of }, can't be
found.  And, a wrapper for each of the three descrambling operations has
been registered.  In any event, the number of unique wrapper(s) registered
will be reported.

After a success above, YTConfig reprocesses its white-space-free copy of
the descrambling function definition it abandoned, far above.  This time,
it parses the definition's text in a loop containing the following body:

        Locate the next period .

        Expect one of the following sequences:
                .join(              >>  Successful conclusion.
                .<reversewrapper>(  >>  Emit r .
                .<slicewrapper>(    >>  Look for ,nn) and emit snn .
                .<swapwrapper>(     >>  Look for ,nn) and emit wnn .
                .split(             >>  Continue.
                .<other>            >>  Non-successful conclusion.

        Otherwise, error conclusion.

Finally, YTConfig reports success or failure and displays any SET command
it has developed.  That gets written in the output file, as well.


Installation and Operational Considerations

PKUNZIP YTCRAC50.ZIP
into an empty directory.  Copy the resulting files to where you want them.
You may as well put YTCRACK.EXE and YTCONFIG.EXE in a directory on your
Path.  There is nothing to configure, as a part of installation.
However, YTConfig is to be run whenever a new signature descrambling key is
needed for YTCrack.  This is discussed further in the Overview section.

When it comes to running YTConfig's .BAT output file, for updating the DOS
Environment, some DOS limitations may become apparent:  First, the DOS
"master" or "command-level" Environment won't get updated by a SET command
issued while "shelled-out" from a program.  Second, the Environment
won't get updated if there isn't enough room for the SET command's string.
These issues are especially likely to be encountered when working in a
Windows DOS window.

In general, some bypasses for these issues are:  Quit or exit the browser
or other program you might be shelled-out from before attempting to run
YTConfig's .BAT output file.  Arrange for an Environment space margin in
your CONFIG.SYS file.  Or, add a DUMMY Environment string to your
Windows DOS window startup definition.  Then, command:
SET DUMMY=
to free up the DUMMY Environment string's space just before running
YTConfig's .BAT output file.  In a Windows XP (SP2) DOS window, I find that
I also may have to view YTConfig's .BAT output file and manually type its
SET command into the command interpreter.
 

Acknowledgements and Copyright Notices

YTConfig owes its existence to Jamie Zawinski, as well as to its author.
The author's understanding of YouTube's video player JavaScript developed
starting from Jamie's youtubedown perl script.  It is available from:
http://www.jwz.org/hacks/youtubedown

Also, I am grateful to Steven Lustig for bringing youtubedown to my
attention.

YTCONFIG.EXE v0.50b and the libraries, materials, and tools used to make it
contain the following Copyright notices:

Borland C++ - Copyright 1991 Borland Intl.

LZEXE.EXE  Version 0.91   (c) 1989 Fabrice BELLARD


Disclaimer

This software is published in the hope that it will be useful, but without
any warranty;  without even the implied warranty of merchantability or
fitness for a particular purpose.


Fred C. Macall  K8GIV
1019 Pennfield Road
Cleveland Heights, Ohio, 44121, U.S.A.
(216) 382-3415
For e-mail contact, run YTConfig .
http://users.ohiohills.com/fmacall/

26 January 2019
