y1z2g3 / owasp-esapi-cplusplus Goto Github PK

On 18 August 2011 09:27, Jeffrey Walton <[email protected]> wrote:
> On Thu, Aug 18, 2011 at 4:21 AM, Jonathan Wakely <[email protected]> 
wrote:
>> On 18 August 2011 08:55, Jeffrey Walton wrote:
>>> Hi Jonathan,
>>>
>>> Sorry to go offlist, but its off topic.
>>
>> This is a slightly trickier one.  The problem is in an incomplete
>> C++0x implementation in libstdc++ 4.6, but there's a simple
>> workaround, full details to follow shortly ...
> OK. I found I could work around with the following. I wanted to make
> sure I was not screwing it up.
>
> Jeff
>
> // construction/destruction
> // inline void construct(pointer p, const T& t) { new(p) T(t); }
> // Added for Fedora 15/GCC 4.6. Was the previous construct non-compliant?
> inline void construct(pointer p, const T& t = T()) { new(p) T(t);
> inline void destroy(pointer p) { p->~T(); }
>

Your allocator is probably OK now you've made it CopyConstructible but there's 
a problem when using custom allocators with the libstdc++ from GCC 4.6

I'm part way through writing a journal article for the ACCU explaining the 
allocator changes in C++11, as I've not seen anyone try to document them, so 
they must be inferred from the C++ standard.  I'll let you know if/when I 
finish writing that.  Anyway ....

In C++03 the signature for std::allocator<T>::construct is:

void construct(pointer p, const T& t)
{
 ::new ((void*)p) T(t);
}

And custom allocators must have the same signature.

In C++11 std::allocator changes it to a variadic template (and note that the 
type constructed is not necessarily the same as the allocator's value type, 
it's deduced from the pointer type passed in as the first parameter):

template<typename T2, typename... Args>
void construct(T2* p, Args&&... a)
{
 ::new ((void*)p) T2(std::forward<Args>(a)...);
}

Custom allocators are not required to have the same signature, in fact in C++11 
they are not required to provide a 'construct' member at all.

In order to retain backward compatibility with custom allocators using the old 
signature, or with new C++11 allocators with no 'construct' there is a new 
class template allocator_traits, which provides a fallback definition, which 
should be used like so:

zallocator<T> z;
typedef allocator_traits<zallocator<T>> traits;
T* buf = traits::allocate(z, 3);
traits::construct(z, buf+0, t);
traits::construct(z, buf+1);
traits::construct(z, buf+2, 1, 2, 3);

The first construct call would use your zallocator's 'construct' member. The 
second and third would use the default definition provided by allocator_traits, 
because your zallocator doesn't have a suitable 'construct' member.  This makes 
everything work perfectly, whether using C++03-style allocators or C++11-style 
allocators.

The problem is that the libstdc++ in GCC 4.6 doesn't have allocator_traits, so 
the calls above would be done by calling your allocator's construct directly:

zallocator<T> z;
T* buf = z.allocate(3);
z.construct(buf+0, t);
z.construct(buf+1);
z.construct(buf+2, 1, 2, 3);

Obviously this fails if zallocator only provides the C++03 interface, which is 
the error you're getting.

I've added allocator_traits so it will be in GCC 4.7, but until then the 
workaround would be to add this to your allocator:

#if defined(__GXX_EXPERIMENTAL_CXX0X__) \
 && __GNUC__ == 4 && __GNUC_MINOR__ < 7

template<typename U, typename... Args>
void construct(U* p, Args&&... a)
{
 ::new ((void*)p) U(std::forward<Args>(a)...);
}

template<typename U>
void destroy(U* p)
{
 delete p;
}

#endif

That should let your allocator work with any code (such as libsdc++ in gcc 4.6) 
which incorrectly relies on the C++11 allocator interface instead of making use 
of allocator_traits

The Intel compiler has flagged a few functions for warning 1125, "entity-kind 
'entity' is hidden by 'entity' -- virtual function override intended?". Please 
verify the intentions. See http://software.intel.com/en-us/articles/cdiag1125/.

My apologies - I don't know what's supposed to happen and what is needed for 
behavior and functionality.

Jeff

icpc -DNDEBUG=1 -g3 -ggdb -O2 -DSAFEINT_DISALLOW_UNSIGNED_NEGATION=1 -pipe 
-std=c++0x -Wall -wd1011 -D_REENTRANT -I. -I./esapi -I./deps 
-I/usr/local/include -fpic -c src/reference/DefaultValidator.cpp -o 
src/reference/DefaultValidator.o
./esapi/reference/validation/BaseValidationRule.h(69): warning #1125: function 
"esapi::ValidationRule::setEncoder(esapi::Encoder &)" is hidden by 
"esapi::BaseValidationRule::setEncoder" -- virtual function override intended?
        virtual void setEncoder(const Encoder &) =0;
                     ^

./esapi/reference/validation/BaseValidationRule.h(94): warning #1125: function 
"esapi::ValidationRule::whitelist(const std::string &, std::set<char, 
std::less<char>, std::allocator<char>>)" is hidden by 
"esapi::BaseValidationRule::whitelist" -- virtual function override intended?
        virtual std::string whitelist(const std::string &, const std::set<char> &);
                            ^

icpc -DNDEBUG=1 -g3 -ggdb -O2 -DSAFEINT_DISALLOW_UNSIGNED_NEGATION=1 -pipe 
-std=c++0x -Wall -wd1011 -D_REENTRANT -I. -I./esapi -I./deps 
-I/usr/local/include -fpic -c src/reference/validation/BaseValidationRule.cpp 
-o src/reference/validation/BaseValidationRule.o
./esapi/reference/validation/BaseValidationRule.h(69): warning #1125: function 
"esapi::ValidationRule::setEncoder(esapi::Encoder &)" is hidden by 
"esapi::BaseValidationRule::setEncoder" -- virtual function override intended?
        virtual void setEncoder(const Encoder &) =0;
                     ^

./esapi/reference/validation/BaseValidationRule.h(94): warning #1125: function 
"esapi::ValidationRule::whitelist(const std::string &, std::set<char, 
std::less<char>, std::allocator<char>>)" is hidden by 
"esapi::BaseValidationRule::whitelist" -- virtual function override intended?
        virtual std::string whitelist(const std::string &, const std::set<char> &);
                            ^

KeyDerivationFunction::computeDerivedKey has a possible security level 
mismatch. See the discussion "KeyDerivationFunction: key derivation key size or 
range?" at 
https://lists.owasp.org/pipermail/owasp-esapi-c++/2011-August/000115.html.

ESAPI_ASSERT uses fprintf rather than std:cerr

On Fri, Aug 5, 2011 at 12:45 AM, Jeffrey Walton <[email protected]> wrote:
> On Fri, Aug 5, 2011 at 12:36 AM, Kevin W. Wall <[email protected]> wrote:
>> Why not use 'debug' for all tests?
> OK, it can be default if you prefer.

At least for the near term. Most of the time, the purpose of writing and
running tests goes hand-in-glove with debugging your code and/or
the tests themselves, so I think debug makes sense for the default
case here.

>> Why would you ever compile tests w/out debug?
> To track down optimization bugs.

OK, valid point...assuming that we have suitable tests. Also, debugging
(or at least debug output) can mask race conditions, so I think you make
a legitimate point. But I think the normal case should still be to
compile with debugging enabled.

>> As for libesapi++.so itself, there can be a
>>
>>    make production
>>
>> which just recursively calls make with RELEASE_BUILD=1 on the command line
>> and the default would be to build libesapi++.so with 'debug' enabled.
> How about `make release`?

Yes; +1; I like 'make release' much better.

>> That's going
>> to be the normal mode for us, probably even for the Release Candidate builds.
> Forgive my ignorance. `make production` should set `RELEASE_BUILD=1`
> but create libesapi++.so with 'debug' enabled?
>
> I'm probably adding to the confusion. A Debug Build has two things in my eye:
>  * no optimizations (O0)
>  * maximum degug info (g3)

AND (IMO), probably -DDEBUG as well.

> A Release build is the opposite:
>  * moderate optimizations (O2)
>  * minimum degug info (g)

OK, I'll buy that.  And probably -DNDEBUG for *final*
releases builds (but NOT for release candidate builds).

> So do you want moderate optimizations and maximum debug info?

Yes.

Need to move to ESAPI exceptions eventually

CryptoHelper needs implementations and test cases for:
* generateSecretKey (needs KeyGenerator factory)
* isCombinedCipherMode (needs encoding format)
* isAllowedCipherMode (needs encoding format)
* isMACRequired (needs CipherText class)
* isCipherTextMACvalid (needs CipherText class)

MessageDigest needs the following implemented when a binary string type is 
implemented.

byte[] MessageDigestImpl<HASH>::digest(byte input[], size_t size)

Oops... three cases where the comment does not hold. At minimum, it might be 
appropriate to spin based on case 3's min/avg/max length.

if ( b1 == b2 ) {
  return true;
}
if ( b1 == null || b2 == null ) {
  return (b1 == b2);
}
if ( b1.length != b2.length ) {
  return false;
}

int result = 0;
// Make sure to go through ALL the bytes. We use the fact that if
// you XOR any bit stream with itself the result will be all 0 bits,
// which in turn yields 0 for the result.
for(int i = 0; i < b1.length; i++) {
  // XOR the 2 current bytes and then OR with the outstanding result.
  result |= (b1[i] ^ b2[i]);
}
return (result == 0) ? true : false;

Near line 90 in the implementation:

// Consistency with Java implementation. This class needs to wire-up a context.
const std::string& label = purpose;
const std::string context;

KeyDerivationFunction::computeDerivedKey is missing Known Answer Tests and 
should be cross validated against the Java gear