adacore / libadalang Goto Github PK

For example

for It in My_Vector.Iterate loop
   ...
end loop;

For the moment we have a - sub-optimal, not really documented API - for name resolution results, namely:

p_ref_val is a property callable on any node, that will return the referenced declaration if applicable.
p_type_val is a property callable on expressions, that will return the type of the expression.

The problem is that for those to work, for the moment the user needs to first call resolve_names on the closest parent that is a complete context for name resolution.

Opening this issue with following goals:

• Call resolve_names automatically
• Rename p_ref_val into p_referenced_decl
• Rename p_type_val into p_type

when parsing a file, the "text" member applied on root node provides the full source code, including comments, except for the leading comments (the ones before the first valid Ada token).

Again, I can workaround it by managing my list of lines in another buffer.

When creating a newtype from an enum, we don't resolve enum literals. There are two problems here:

1. We need to re-introduce enum literals, probably through a referenced_env
2. Typing needs to consider their new types

A treatment similar to the one for inherited primitives seems necessary.

type A is (B, C, D);

type B is new A range C .. D;

-- Does not yet resolve
Inst : B := C;

For the moment we're not name resolving anything in generic instantiations.
Opening this issue to track the work of implementing what's needed to resolve
names in generic instantiations.

As explained here, it would be useful to have properties on entities to retrieve the list of generic instanciation that lead to the creation of the said entity.

The following script:

import libadalang as lal
ctx = lal.AnalysisContext()
u = ctx.get_from_buffer(
'foo.adb',
'''
package body Foo is
    procedure Bar (I : Integer) is
    begin
        null;
    end Bar;
end Foo;
''')

p = u.root.find(lal.ParamSpec)
print('{} -> {}'.format(p, p.p_semantic_parent))

Gives:

<ParamSpec ["I"] 3:20-3:31> -> <PackageBody ["Foo"] 2:1-7:9>

I would expect it to print instead:

<ParamSpec ["I"] 3:20-3:31> -> <SubpBody ["Bar"] 3:5-6:13>

Raphaël, can you please have a look? Thank you in advance!

For the moment we do not resolve anything with regard to protected objects and
types.

This linter warned about the assignment on the next line, saying the value is never used:

        Next_Local_Start_Time := Truncate_To_Day(Next_Local_Start_Time) + Input_Start_Hour;
     end if;

     Next_Start_Time := Time_Zones.Convert(T => Next_Local_Start_time,

^ But it was used immediately after. However the identifiers were capitalized differently (..._Time vs ..._time) which must have confused the linter.

Hi, I'm looking at replacing one of Rapita's ASIS-based tools with libadalang. I have been pleased with the ease of using libadalang from Ada and also from Python. I have already been able to capture the structure of the source code, enough to implement statement coverage.

However one important requirement for our tool is that it must be able to go from a usage to a definition. This needs to work for variable references, type references, packages, instantiations, procedure calls. This is no problem in ASIS because the API provides functions such as "Corresponding_Declaration". But I can't figure out how to do the same thing in libadalang.

For example, here is a short Ada program:

  1 
  2 procedure demo is
  3 
  4    procedure proc is
  5    begin
  6       null;
  7    end proc;
  8 
  9    type int1_type is new Natural;
 10    type int2_type is new int1_type; -- should be linked to "int1_type" declaration
 11 
 12    var : int2_type; -- should be linked to "int2_type" declaration
 13 
 14 begin
 15    proc;           -- should be linked to "proc" declaration
 16    var := var + 1; -- should be linked to "var" declaration
 17 end demo;

Suppose I start with the libadalang node representing the call statement on line 15. How do I get to the procedure declaration on line 4? Same question for the "var" identifiers on line 16 (to line 12), and for the type declaration on line 10 (to line 9). Is it possible to do this? If not, are there plans to implement this?

I tried to insert statements before a null; statement that was indented by 2 tabulation chars, and the insertion position that was given was incorrect, resulting in an insertion in the middle of the null word.

workaround: replaced tabulation chars by 4 spaces (any number would be OK) and it inserted the statement properly, so not blocking but you may want to look into this.

For the moment, package bodies are always rooted in their package declaration.
However, this is not enough:

package Foo is
   package Bar is
       A : Integer := 12;
   end Bar
end Foo;

package body Foo is
   B : Integer := 15;
   package Body Bar is
      O : Integer :=  A + B;
   end Bar;
end Foo;

Here, to resolve O's default expression, we need visibility on both Foo's
body and Bar's decl.

Also, both links needs to be transitive, so using a non-transitive reference
won't be enough.

The result is that, from the lookup point of O, we're turning the env tree into
a DAG, and with our naïve lookup algorithm, some elements will be returned
twice. We will live with it for now, but we will need a fix at some point.

python ada/manage.py generate

gives me:

�[95mGenerating source for libadalang ...�[0m
�[94mCompiling the grammar...�[0m
File "�[96mK:\temp\ada\language\ast.py�[0m", line �[96m3275�[0m, in BlockStmt._env_key_44
line �[96m3282�[0m, in FieldAccess expression
�[1m�[91mError�[0m: SymbolType values have no field (accessed field was to_array)
�[91mErrors, exiting�[0m

For the moment aggregates for discriminated records are completely wrong.
Notably the order of fields is wrong. For example, this simple code will not resolve:

procedure Fail is
    type Rec (N : Boolean) is record
        A, B : Integer;
    end record;

    Inst : Rec := (True, 12, 15);
begin
    null;
end Fail;

We want people to start using the semantic part of the public API soon. As part of this, this part of LAL needs to be improved an reviewed.

Is it possible to obtain the fully qualified name of a declaration ?

For example, when parsing the 'Time' type in 'Ada.Calendar', is it possible to get 'Ada.Calendar.Time_Rep' for the parent type, instead of simply 'Time_Rep' ?

We need to implement name resolution for formal packages.

Associated RM page: http://www.adaic.org/resources/add_content/standards/05rm/html/RM-12-7.html

Given the following package:

with Ada.Calendar;   use Ada.Calendar;
package Main is
   package Nested is
      subtype My_Time is Time;
   end Nested;
   use Nested;
   type T is record
      Field1 : My_Time;
   end record;
end Main;

I am using python code like:

for d in unit.root.findall(SubtypeDecl):     # declaration of MyType
      print decl.f_subtype.f_name   # <Identifier>         
      print decl.f_subtype.f_name.p_referenced_decl     # Internal error

It raises an internal error:

  File "/usr/local/stow/libadalang/python/libadalang.py", line 785, in p_referenced_decl
    return AdaNode._wrap(self._eval_field(Entity._c_type(), _ada_node_p_referenced_decl))
  File "/usr/local/stow/libadalang/python/libadalang.py", line 1178, in _eval_field
    raise PropertyError(*exc.contents._wrap().args)
libadalang.PropertyError: raised LIBADALANG.ANALYSIS.PROPERTY_ERROR : dereferencing a null access

Hi,

I'm doing some experiments on fuzzing libadalang, to see how robust the parser is.

I use the "parse" main program :

parse -C -P -f my_file

and am trying to trigger a top-level (uncaught) exception.

I have some test cases that trigger a constraint error at libadalang-sources.adb:40 range check failed.

The sha-1 I'm testing against : f5a3ba0 .

Test cases are attached here ticket-constraint-error-libadalang-sources-40.zip

Reproducer:

procedure Ex1 is
   type Point is record
      x : Integer;
      y : Integer;
      z : Integer;
   end record;
   p : Point;
begin
   p := (others => 42);
end Ex1;

Calling p_expression_type on the rhs of the others designator (42 here) returns None.

Trying to build the libadalang by running the sudo python ada/manage.py build I get the following error:

Build Failed: [Errno2] No such file or directory

I have downloaded and installed gnat ada 2012 from libre ada and quex 0.65.4

Hi, I'm trying to fuzz the libadalang parser, and found a (very simple) case that if launched with the "parse" main program (parse -C -P -f my_file), breaks with a constraint error at :

gnatcoll-iconv.adb:116:44 index check failed index 3 not in 1..2

The example (very simple) is attached here : libadalang-parse-gnat-icoll-constraint-error.zip

Of course, it's not valid code and it's an extreme case (that's what you get with fuzzing)... If you try to open it in some file editors, it will look empty. According to wc -c there are two characters, and according to "less" they are : <FE><FF>

The sha-1 I'm testing against : f5a3ba0 .

We're missing name resolution for aggregates of variant part record types. If we implement the full feature, this requires a full static expression evaluator:

type My_Rec (Disc : Integer) is record
    case Disc is 
        when 1 => ...
        when 2 => ...
        when others => ...
    end case;
end record;


Inst : My_Rec := (1 + 2 / 3 + 1000, ...);

However in a first iteration we'll only care about enum literals, expanding the evaluator as needed.

Installing libadalang
$ virtualenv env $ source env/bin/activate $ pip install -r REQUIREMENTS.dev

Generates the following error in regards langkit setup

Downloading/unpacking Mako==1.0.1 (from -r REQUIREMENTS.dev (line 1))
Downloading Mako-1.0.1.tar.gz (473kB): 473kB downloaded
Running setup.py (path:/tmp/pip-build-BO9r_2/Mako/setup.py) egg_info for package Mako

warning: no files found matching '*.xml' under directory 'examples'
warning: no files found matching '*.mako' under directory 'examples'
warning: no files found matching 'ez_setup.py'
no previously-included directories found matching 'doc/build/output'

Downloading/unpacking PyYAML==3.11 (from -r REQUIREMENTS.dev (line 2))
Downloading PyYAML-3.11.zip (371kB): 371kB downloaded
Running setup.py (path:/tmp/pip-build-BO9r_2/PyYAML/setup.py) egg_info for package PyYAML

Downloading/unpacking Sphinx==1.3.1 (from -r REQUIREMENTS.dev (line 3))
Downloading Sphinx-1.3.1-py2.py3-none-any.whl (1.3MB): 1.3MB downloaded
Downloading/unpacking coverage==3.7.1 (from -r REQUIREMENTS.dev (line 4))
Downloading coverage-3.7.1.tar.gz (284kB): 284kB downloaded
Running setup.py (path:/tmp/pip-build-BO9r_2/coverage/setup.py) egg_info for package coverage

warning: no previously-included files matching '*.pyc' found anywhere in distribution

Downloading/unpacking enum==0.4.6 (from -r REQUIREMENTS.dev (line 5))
Downloading enum-0.4.6.tar.gz
Running setup.py (path:/tmp/pip-build-BO9r_2/enum/setup.py) egg_info for package enum

Downloading/unpacking enum34==1.1.2 (from -r REQUIREMENTS.dev (line 6))
Downloading enum34-1.1.2.tar.gz (46kB): 46kB downloaded
Running setup.py (path:/tmp/pip-build-BO9r_2/enum34/setup.py) egg_info for package enum34

Downloading/unpacking psutil==3.4.2 (from -r REQUIREMENTS.dev (line 7))
Downloading psutil-3.4.2.tar.gz (274kB): 274kB downloaded
Running setup.py (path:/tmp/pip-build-BO9r_2/psutil/setup.py) egg_info for package psutil

warning: no previously-included files matching '*' found under directory 'docs/_build'

Downloading/unpacking sphinx-rtd-theme==0.1.9 (from -r REQUIREMENTS.dev (line 8))
Downloading sphinx_rtd_theme-0.1.9-py2-none-any.whl (693kB): 693kB downloaded
Downloading/unpacking funcy==1.7.1 (from -r REQUIREMENTS.dev (line 9))
Downloading funcy-1.7.1.tar.gz
Running setup.py (path:/tmp/pip-build-BO9r_2/funcy/setup.py) egg_info for package funcy

Obtaining langkit from git+https://github.com/AdaCore/langkit.git#egg=langkit (from -r REQUIREMENTS.dev (line 10))
Cloning https://github.com/AdaCore/langkit.git to ./env/src/langkit
Running setup.py (path:/home/zboll/git/libadalang/env/src/langkit/setup.py) egg_info for package langkit
error in Langkit setup command: package_data must be a dictionary mapping package names to lists of wildcard patterns
Complete output from command python setup.py egg_info:
error in Langkit setup command: package_data must be a dictionary mapping package names to lists of wildcard patterns

---

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /home/zboll/git/libadalang/env/src/langkit
Storing debug log for failure in /home/zboll/.pip/pip.log

Running nameres --all --with-default-project pck.ads on the following source file:

with GNAT.Regexp;

package Pck is
   procedure P1 (I : Integer);
   procedure P2 (R : GNAT.Regexp.Regexp);

   procedure P3 (IA : access Integer);
   procedure P4 (RA : access GNAT.Regexp.Regexp);
end Pck;

Yields odd results:

• There is no resolution for P1 nor P2.
• Integer from P3 is correctly resolved.
• Various intermediate nodes in access GNAT.Regexp.Regexp are not resolved:

***************************************************************

Expr: <DottedName 8:30-8:48>
  references: <TypeDecl ["Regexp"] 103:4-103:27>
  type:       None
Expr: <DottedName 8:30-8:41>
  references: None
  type:       None
Expr: <Id "GNAT" 8:30-8:34>
  references: None
  type:       None
Expr: <Id "Regexp" 8:35-8:41>
  references: None
  type:       None
Expr: <Id "Regexp" 8:42-8:48>
  references: <TypeDecl ["Regexp"] 103:4-103:27>
  type:       None

It seems that elsif isn’t covered in an if-expression:

Traceback (most recent call last):
  File "check_test_not_null.py", line 317, in <module>
    main(parser.parse_args())
  File "check_test_not_null.py", line 313, in main
    do_file(f)
  File "check_test_not_null.py", line 308, in do_file
    explore(f, subp)
  File "check_test_not_null.py", line 295, in explore
    traverse_subp_body(subp, {})
  File "check_test_not_null.py", line 293, in traverse_subp_body
    traverse(sub, nulls)
  File "check_test_not_null.py", line 287, in traverse
    traverse(sub, nulls)
  File "check_test_not_null.py", line 287, in traverse
    traverse(sub, nulls)
  File "check_test_not_null.py", line 218, in traverse
    traverse(sub, nulls)
  File "check_test_not_null.py", line 287, in traverse
    traverse(sub, nulls)
  File "check_test_not_null.py", line 245, in traverse
    for sub in node.f_elsif_list:
AttributeError: 'IfExpr' object has no attribute 'f_elsif_list'

procedure Ex1 is
   type My_Variant(b : Boolean) is record
      case b is
         when True =>
            x : Integer;
         when False =>
            y : Boolean;
      end case;
   end record;
   x : My_Variant(False);
begin
   x.y := x.b;
end Ex1;

When trying p_referenced_decl on the type_expr of the discriminant spec I get None. I need to manually call p_resolve_names beforehand to get the Boolean type decl.

We don't yet resolve aggregates for multidimensional arrays

I set it up like in the examples, but when I open a file with degrees (°) even in comments, libadalang refuses to analyze it.

If I remove this char, it works properly (not an issue at this point, I can workaround)

BTW in case you did not get it I'm JF Fabre from Thales Avionics and I'm more than pleased with libadalang at this point.

Running the following script reveals a hole in the name resolution:

import libadalang as lal
ctx = lal.AnalysisContext()
ctx.get_from_buffer('foo.ads', '''
package Foo is
    procedure P;
end Foo;
''')
u = ctx.get_from_buffer('foo.adb', '''
package body Foo is
    procedure P is null;
end Foo;
''')

p = u.root.find(lal.Identifier)
print('{} -> {}'.format(p, p.p_referenced_decl))

That yields a Property_Error whereas it should print the node for “package Foo”.

Discriminant constraints are not yet resolved.

type A (C : Character; B : Boolean) is null record;

Inst : A ('l', True);
--       ^ This part needs name res

This is the continuation of issue #16 , but for synchronized interfaces, and protected/task types.

Has anybody tried building on Windows? I get the following output in a Windows command shell which errors in trying to execute quex-exe.py. Perhaps it is all the double slashes since it seems like this script expects to be running in a *nix environment?

c:\Users\Matt\Downloads\AdaCore-Download-2017-11-16_1312\libadalang-master>c:\Python27\python.exe .\ada\manage.py generate
�[95mGenerating source for libadalang...�[0m
�[94mCompiling the grammar...�[0m
�[94mCompiling properties...�[0m
�[94mPrepare code emission...�[0m
�[94mFile setup...�[0m
�[94mGenerating sources... �[0m
�[94mCompiling the quex lexer specification�[0m
command line: Command line option '--token-memory-management-by-user' is ignored.
command line: User token memory management option no longer available.
command line: Last version of Quex supporting this option is version 0.67.4. Please, visit
command line: http://quex.org for further information.
�[1m�[91mError�[0m: Command '['c:\Python27\python.exe', 'C:\Program Files (x86)\quex/quex-0.67.5/quex-exe.py', '-i', 'c:\Users\Matt\Downloads\AdaCore-Download-2017-11-16_1312\libadalang-master\build\include\libadalang\ada.qx', '-o', 'quex_lexer', '--buffer-element-size', '4', '--token-id-offset', '0x1000', '--language', 'C', '--no-mode-transition-check', '--single-mode-analyzer', '--token-memory-management-by-user', '--token-policy', 'single', '--token-id-prefix', 'ADA_TKN_']' returned non-zero exit status -1
�[91mInternal error! Exiting�[0m

Name resolution is very slow at the moment, and at least 80% of the time is spent in scopes lookups.

One hope was that memoization of properties would be enough for that purpose, but it turns out not to be.

The plan is to implement caches in lexical environments. Lex env lookups need to be refactored to make that possible.

We still have missing language defined attributes, as well as GNAT specific ones. Opening this issue to track implementation of missing ones.

Hi,

Minor issue when running your script install-lal-and-deps.sh in Debian (or Ubuntu): Sourcing virtualenv fails, because /bin/sh is pointing to /bin/dash, where the command "source" is unknown.
Workaround: Change the interpreter to /bin/bash.

The following script:

import libadalang as lal
ctx = lal.AnalysisContext()
u = ctx.get_from_buffer(
'foo.ads',
'''
package Foo is
    type Rec (N : Natural);
private
    type Rec (N : Natural) is null record;
end Foo;
''')

p = u.root.find(lal.DiscriminantSpec)
print('{} -> {}'.format(p, p.p_semantic_parent))

Gives:

<DiscriminantSpec ["N"] 3:15-3:26> -> <PackageDecl ["Foo"] 2:1-6:9>

I would expect it to print instead:

<DiscriminantSpec ["N"] 3:15-3:26> -> <IncompleteTypeDecl ["Rec"] 3:5-3:28>

Raphaël, can you please have a look? Thank you in advance!

In Ada, using a qualified name in a package body or in a private part should resolve to the "most visible" part. It should also make elements from the private and public part visible:

package A is
   procedure Foo;
   Spec_Int : Integer := 10;
private
   Private_Int : Integer := 11;
end A;

package body A is
   Body_Int : Integer := 9
   procedure Foo is
   begin
      Put_Line (Integer'Image (A.Spec_Int + A.Private_Int + A.Body_Int));
   end Foo;
end A;

This does not yet work in LAL, and will require alterations to the way lexical envs work.

ASIS is widely used for static analysis of Ada code, and is an ISO standard (ISO/IEC 15291:1995).

The README does not make it clear why I'd use libadalang instead of ASIS, and it really needs to provide that info. The ASIS spec may not include some of the latest Ada features, but I presume that they'd be easy to add. Why would I use libadalang instead?

Running nameres --with-default-project pck.adb with the following sources:

--  pck.ads
with Ada.Unchecked_Deallocation;

package Pck is

   type String_Access is access all String;
   type Integer_Access is access all Integer;

   procedure Free is new Ada.Unchecked_Deallocation (String, String_Access);
   procedure Free is new Ada.Unchecked_Deallocation (Integer, Integer_Access);

   procedure Foo;

end Pck;

--  pck.adb
package body Pck is

   procedure Foo is
      IA : Integer_Access := new Integer'(1);
      SA : String_Access := new String'("hello");
   begin
      Free (IA);
      pragma Test_Statement;

      Free (SA);
      pragma Test_Statement;
   end Foo;

end Pck;

Yields unexpected results:

Expr: <Id "Free" 7:7-7:11>
  references: <| GenericSubpInternal ["Ada.Unchecked_Deallocation"] 20:1-20:55 [pck.ads:9:4] |>
  type:       None
[…]
Expr: <Id "Free" 10:7-10:11>
  references: <| GenericSubpInternal ["Ada.Unchecked_Deallocation"] 20:1-20:55 [pck.ads:8:4] |>
  type:       None

We would have expected instead references to the GenericSubpInstantiation nodes in pck.ads.

As explained in the RM: http://www.ada-auth.org/standards/12rm/html/RM-4-1-5.html

We're missing name resolution for interface types.

Interface types have no components, but they can have associated subprograms, that can be called by dot notation or by direct calls.

Also, protected and task types can inherit from interfaces, but this will be kept for a separate issue, given that it's a pretty different feature.

Running nameres --all pack.ads with the following file:

package Pack is
   function F return Natural;

   type Ints is array (Natural range <>) of Integer;
   type Bounded_Ints (Max_Size : Natural) is record
      Length : Natural := 0;
      Objs   : Ints (1 .. Max_Size);
   end record;

   type Ints_Doubled is array (1 .. 2) of Bounded_Ints (F);
end Pack;

Shows missing resolved references.

Expr: <Id "Max_Size" 7:27-7:35>
  references: None
  type:       None
Expr: <Id "Bounded_Ints" 10:43-10:55>
  references: None
  type:       None
Expr: <Id "F" 10:57-10:58>
  references: None
  type:       None

Raphaël, can you have a look at this, please? Thank you in advance!

python is great for pragmatic shortcuts, like getting the last item of list by doing my_list[-1]

unfortunately, it doesn't work with AdaNodeLists as provided by libadalang so I have to do: node.children[len(node.children)-1] and I hate that :)

We need a static expression evaluator in LAL, the most immediate use case being name resolution of aggregates for records with variant parts (#19)

As in

Bla:
for A in 1 .. 1000 loop
   exit Bla when A > 10;
   --    ^ Resolve this name
end loop Bla;

I avoided packaging libadalang for a long time because of this line:

Libadalang is still pre-alpha software! None of its APIs are stable, the shape of the abstract syntax tree is not yet stable, and most of its features are either not stable or not fully implemented.

Recently, I discovered that GPS 2017 required it and was forced to import "pre-alpha" software. I'm guessing that because it's used in production software, libadalang is no longer "pre-alpha".

It's my opinion this line is damaging if it's not true. I know it affected my decisions. I would recommend updating this line ASAP. It's unfortunate that it made it into branch gpl-2017.

procedure Test is
   subtype Natural is Integer range 0 .. Integer'Last;
   x : Integer := Natural'First;
begin
   null;
end Test;

Getting a libadalang.PropertyError: raised LIBADALANG.ANALYSIS.PROPERTY_ERROR : dereferencing a null access error when trying to get property p_expression_type on the literal 0 appearing in the subtype declaration.

We're missing implementation of use type/use all type in name resolution.

One subtlety is that use all type also makes visible all dottable subprograms, including the ones defined via a 'Class parameter:

package TT is
    type Foo is tagged null record;
    
    procedure Frobulize (Self : Foo'Class);
end TT;

use all type TT.Foo;
I : Foo;
..

Frobulize (I);

Here is the associated RM page: http://www.ada-auth.org/standards/12rm/html/RM-8-4.html

Hello,

We tried to use libadalang to generate lexical environnement but when we run it, populate_lexical_env crashes for 2 on 4 test files (types_bidon.ads which defines external types and test-startup.adb which is a separate of test.adb).

Here are the files: test files, executable and the way to run it.
parsing_test.zip

Do you have any idea of the source of the problem?

Regards,