Yes. Less of a concern, sure, but what you are describing is a cipher-text-only attack. Absolutely and without a doubt still possible.

I want to split your question into two cases:

1) You write your own implementation of a standardized cipher.

Sure, in this case I agree with you: as long as your implementation produces exactly the same output as the reference implementation, and there is no way for attackers to perform side-channel attacks, then it's not really "rolling your own" because you are using an algorithm that is known to be secure.

2) You invent your own cipher or algorithm

Here's where bad things happen. Generally, the attack is frequency analysis - for example if I know that your texts are written in English, then I can start analyzing your ciphertexts looking for patterns that match the distribution of letters in English. If your texts are in XML format, then I can look for the general structure of nested XML blocks, etc.

Never underestimate the cleverness of professional cryptanalysts. Also, never forget Schneier's Law:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

Just because you can't break it, doesn't mean nobody can.

Yes. Less of a concern, sure, but what you are describing is a cipher-text-only attack. Absolutely and without a doubt still possible.

I want to split your question into two cases:

1) You write your own implementation of a standardized cipher.

Sure, in this case I agree with you: if you have made an implementation of an algorithm that's faithful to the spec, then your ciphertext by itself will be no easier to break than the ciphertext of any other compliant implementation.

I'd argue you're not really "rolling your own".

Being resistant to side-channel attacks is really more of "hardening" than "rolling your own"; every implementation anywhere could use more hardening. Always.

2) You invent your own cipher or algorithm

Here's where bad things happen. Generally, the attack is frequency analysis - for example if I know that your texts are written in English, then I can start analyzing your ciphertexts looking for patterns that match the distribution of letters in English. If your texts are in XML format, then I can look for the general structure of nested XML blocks, etc.

Never underestimate the cleverness of professional cryptanalysts. Also, never forget Schneier's Law:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

Just because you can't break it, doesn't mean nobody can.

Yes. Less of a concern, sure, but what you are describing is a cipher-text-only attack. Absolutely and without a doubt still possible.

Generally, the attack is frequency analysis - for example if I know that your texts are written in English, then I can start analyzing your ciphertexts looking for patterns that match the distribution of letters in English. If your texts are in XML format, then I can look for the general structure of nested XML blocks, etc.

Never underestimate the cleverness of professional cryptanalysts. Also, never forget Schneier's Law:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

Just because you can't break it, doesn't mean nobody can.

Yes. Less of a concern, sure, but what you are describing is a cipher-text-only attack. Absolutely and without a doubt still possible.

I want to split your question into two cases:

1) You write your own implementation of a standardized cipher.

Sure, in this case I agree with you: as long as your implementation produces exactly the same output as the reference implementation, and there is no way for attackers to perform side-channel attacks, then it's not really "rolling your own" because you are using an algorithm that is known to be secure.

2) You invent your own cipher or algorithm

Here's where bad things happen. Generally, the attack is frequency analysis - for example if I know that your texts are written in English, then I can start analyzing your ciphertexts looking for patterns that match the distribution of letters in English. If your texts are in XML format, then I can look for the general structure of nested XML blocks, etc.

Never underestimate the cleverness of professional cryptanalysts. Also, never forget Schneier's Law:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

Just because you can't break it, doesn't mean nobody can.