Remove debris off the green?

Rulefan's explanation is comprehensive. Yes you may remove the debris whether it comprises natural objects like leaves (loose impediments) or artificial ones like crisp packets (movable obstructions). Best to pick them up when off the green rather than brush them away as you might also brush away sand or soil which are not loose impediments except on the green.

The rules are 23-1 as mentioned above for loose impediments and 24-1 for movable obstructions.

By the way, you only have a line of putt when your ball is lying on the green but that's not relevant to your question.