Most of the items drop from SuperUnique monsters (the ones with a unique purple name and a title instead of their affixes below it), so keep an eye out for them, especially for Bounties targeting them!
It appears that most of the materials only drop for level 70 characters.
It appears that these materials can be acquired on any difficulty and game mode.
A Polish communty manager has recently posted exact drop rates from SuperUniques per difficulty here:
| Normal | Hard | Expert | Master | Torment I | Torment II | Torment III | Torment IV | Torment V | Torment VI |
|---|---|---|---|---|---|---|---|---|---|
| 5% | 6% | 7.2% | 8.6% | 10.4% | 12.4% | 14.9% | 17.9% | 21.5% | 25.8% |
Adventurer's Journal and Grisworld's Scribblings can drop for lower level characters, even in vanilla Diablo 3. This is likely due to the fact that they drop from environment objects instead of monsters.