I'm not positive, but I think it's more like they've historically been used both ways, but that a trend towards mandating the alternating ([( pattern and excluding nested parentheses to avoid confusion probably started in the 18th or 19th century, which was the time of the first major trend of writing prescriptive grammar/style guides for English.