How can I add or remove character from a specified line number in shell?

Question

I have a file which I must iterate through several times, running it with python each time. How can I remove or add characters from specific lines in shell?

for an example file ex.file,

$ cat ex.file
stuff
more stuff
more stuff
more stuff
#<- remove this character
<-add a pund sign here
more stuff
more stuff

my desired output would be:

$ cat ex.file
stuff
more stuff
more stuff
more stuff
<- remove this character
#<-add a pund sign here
more stuff
more stuff

Answer 1

To add a # at the start of line 6:

sed '6 s/^/#/' ex.file

To remove the leading # from line 5:

sed '5 s/^#//' ex.file